Pseudomonotone Variational Inequalities: Convergence of the …naelfaro/publis/ElFarouq-JOTA... · 2021. 3. 31. · N. EL FAROUQ 1 Communicated by Z. Q. Luo Abstract. This paper deals

JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS: Vol. 111, No. 2, pp. 305–326, November 2001 ( 2001)

Pseudomonotone Variational Inequalities:Convergence of the Auxiliary Problem Method

N. EL FAROUQ1

Communicated by Z. Q. Luo

Abstract. This paper deals with the convergence of the algorithm builton the auxiliary problem principle for solving pseudomonotone (in thesense of Karamardian) variational inequalities.

Key Words. Variational inequalities, optimization problems, general-ized monotonicity, pseudomonotonicity, convergence of algorithms.

1. Introduction

Let χ be a Hilbert space, let χ ad be a nonempty closed and convexsubset of χ , referred to as the admissible set, and let Ψ be an operator fromχ to itself. We consider the following variational problem: find x* ∈ χ ad suchthat

⟨Ψ(x*), xAx*⟩¤0, ∀ x ∈ χ ad. (1)

In the more general case, where Ψ is a point-to-set operator, the multivaluedvariational inequality problem is stated as follows: find x* ∈ χ ad such that

∃ r* ∈Ψ (x*) s. t. ⟨r*, xAx*⟩¤0, ∀ x ∈ χ ad. (2)

A survey on the variational inequality problem (1) in finite-dimensionalspaces is the paper by Harker and Pang (Ref. 1). In this article, the readerwill find motivations, examples, results, and a vast bibliography.

Various iterative schemes have been proposed for solving the variationalinequality problem. In the present paper, we study the convergence of themethod based on the auxiliary problem principle under a pseudomono-tonicity assumption on the operator. Recently, this property has been theobject of many papers, among other concepts of generalized monotonicity.

1Maıtre de Conferences, Universite Blaise Pascal, Clermont II and LAAS CNRS, Toulouse,France.

3050022-3239011100-0305$19.500 2001 Plenum Publishing Corporation

JOTA: VOL. 111, NO. 2, NOVEMBER 2001306

With no claims of exhaustivity, we quote the works of Karamardian (Ref. 2),Karamardian and Schaible (Ref. 3), Schaible (Ref. 4), Karamardian, Scha-ible, and Crouzeix (Ref. 5), Komlosi (Ref. 6), and Zhu and Marcotte (Ref. 7).The notion of pseudomonotonicity of operators is related closely to that ofpseudoconvexity of functions, introduced by Mangasarian (Refs. 8-9). Sev-eral examples and uses of generalized monotonicity are given in Ref. 10.Results on the existence of solutions to pseudomonotone variational in-equalities can be found, for example, in the works of Karamardian (Ref. 2),Harker and Pang (Ref. 1), Yao (Refs. 11-12), and Crouzeix (Ref. 13).

In particular, we present the convergence results obtained for single-valued variational inequalities involving general operators satisfying somekind of pseudomonotonicity property: the pseudo-Dunn property on theone hand and the strong pseudomonotonicity property on the other hand.Under this latter assumption, we still get the convergence of the auxiliaryproblem method in the case of multivalued variational inequalities. Whenthe variational inequality is a minimization problem and when χ is a finite-dimensional space, a convergence theorem based on the pseudomonotonic-ity of the gradient is also given.

2. Preliminaries

In this section, we give various definitions and basic results in the fieldof generalized monotonicity. Unless we specify explicitly that the operatoris multivalued, it is assumed to be single valued.

Definition 2.1.

(i) Ψ is monotone on χ ad if

∀ x1 , x2 ∈ χ ad, ⟨Ψ(x2)AΨ(x1), x2Ax1 ⟩¤0.

(ii) Ψ is strongly monotone on χ ad if

∃ aH0 s.t., ∀ x1 , x2 ∈ χ ad,

⟨Ψ(x2)AΨ(x1), x2Ax1 ⟩¤ax2Ax1 2.

(iii) Ψ has the Dunn property on χ ad if

∃ AH0 s.t., ∀ x1 , x2 ∈ χ ad,

⟨Ψ(x2)AΨ(x1),x2Ax1 ⟩¤ (1A) Ψ(x2)AΨ(x1) 2.

This property is equivalent to the strong monotonicity of the possibly multi-valued operator ΨA1. It is also referred to as cocoercivity in Ref. 14. When

JOTA: VOL. 111, NO. 2, NOVEMBER 2001 307

AG1, Ψ is said to be firmly nonexpansive (Ref. 15). Moreover, the Dunnproperty is satisfied by the Moreau-Yosida regularized operator of a mono-tone operator (Ref. 16), which is equivalent to the firm nonexpansivenessof the resolvent of a monotone operator (Ref. 15).

(iv) Ψ is pseudomonotone on χ ad if

∀ x1 , x2 ∈ χ ad, ⟨Ψ(x1), x2Ax1 ⟩¤0 ⇒ ⟨Ψ (x2), x2Ax1 ⟩¤0.

Pseudomonotonicity is understood here in the sense of Karamardian (Ref.2), and not in the sense of Brezis (Ref. 17). The latter concerns some topo-logical properties on the operator.

(v) Ψ is strongly pseudomonotone on χ ad (Ref. 3) if

∃ eH0 s.t., ∀ x1 , x2 ∈ χ ad,

⟨Ψ(x1), x2Ax1 ⟩¤0 ⇒ ⟨Ψ (x2), x2Ax1 ⟩¤ex2Ax1 2.

(vi) Ψ has the pseudo-Dunn property on χ ad if

∃ EH0 s.t., ∀ x1 , x2 ∈ χ ad,

⟨Ψ(x1), x2Ax1 ⟩¤0 ⇒ ⟨Ψ (x2), x2Ax1 ⟩¤ (1E ) Ψ(x2)AΨ(x1) 2.

This property is also referred to as pseudococoercivity in Ref. 7.(vii) Ψ is quasimonotone on χ ad if

∀ x1 , x2 ∈ χ ad, ⟨Ψ(x1), x2Ax1 ⟩H0 ⇒ ⟨Ψ (x2), x2Ax1 ⟩¤0.

Remark 2.1. Monotonicity of a multivalued operator on χ ad corre-sponds to

∀ x1 , x2 ∈ χ ad, ∀ r1 ∈Ψ (x1), ∀ r2 ∈Ψ (x2), ⟨r2Ar1 , x2Ax1 ⟩¤0.

The same kind of modification applies to the definitions of strong monoton-icity, pseudomonotonicity, strong pseudomonotonicity, and quasimonoton-icity. However, if we take x1Gx2 in Definition 2.1(iii) (respectivelyDefinition 2.1(vi)), then Ψ(x1)GΨ(x2). This means that, if an operator hasthe Dunn property (respectively the pseudo-Dunn property), then it issingle-valued.

Lemma 2.1.

(i) If Ψ is pseudomonotone, then for all x*1 and x*2 solutions of (1),

⟨Ψ(x*2 ), x*2 Ax*1 ⟩G0. (3)


(ii) If Ψ has the pseudo-Dunn property, then the set

SGΨ(x*) ⟨Ψ(x*), xAx*⟩¤0, ∀ x ∈ χ ad

is a singleton.(iii) If Ψ is strongly pseudomonotone and if problem (1) has a solu-

tion, then it is unique. This result is still valid when the operatoris multivalued.

Proof. Let x*1 and x*2 in χ ad be solutions of (1). Then,

⟨Ψ(x*1 ), x*2 Ax*1 ⟩¤0, (4)

⟨Ψ(x*2 ), x*1 Ax*2 ⟩¤0. (5)

(i) If Ψ is pseudomonotone, then (4) implies

⟨Ψ(x*2 ), x*2 Ax*1 ⟩¤0.

Then, from (5), we get (3).(ii) If Ψ has the pseudo-Dunn property, then

0G⟨Ψ(x*2 ), x*2 Ax*1 ⟩¤ (1E ) Ψ(x*2 )AΨ(x*1 ) 2 .

Therefore, Ψ(x*2 )GΨ(x*1 ).(iii) By using the same reasoning when Ψ is strongly pseudomono-

tone, we get

x*2 Gx*1 .

We illustrate hereafter the relationships between the monotonicityassumption and some generalized monotonicity assumptions:

Strong monotonicity ⇒ Monotonicity ⇐ Dunn property

⇓ ⇓ ⇓Strong pseudomonotonicity ⇒ Pseudomonotonicity ⇐ Pseudo Dunn property

⇓Quasimonotonicity

We also get easily the following result by using the definitions.

Lemma 2.2. If Ψ is strongly pseudomonotone with constant e andLipschitz continuous with constant L, then it has the pseudo-Dunn propertywith constant L2e.

The following examples show that the converse of the implicationsabove is not true and that there is no direct link between monotonicity andstrong pseudomonotonicity.


Example 2.1.

(i) Let Ψ(x)G1Ax, and let χ adG[0, 1]. Ψ is pseudomonotone, butnot monotone. Also, it has neither the pseudo-Dunn property nor the strongpseudomonotonicity property.

(ii) Let Ψ(x)GAx and χ adG[0, +S[. Ψ is quasimonotone, but notpseudomonotone.

(iii) Let Ψ(x)G1x and χ adG[1, +S[. Ψ has the pseudo-Dunn prop-erty with constant 1. It does not satisfy the Dunn property and is not evenmonotone. It is Lipschitz continuous with constant 1, but not stronglypseudomonotone.

(iv) Let Ψ(x)G2Ax and χ adG[0, 1]. Ψ is strongly pseudomonotonewith constant 1. It is not strongly monotone and even not monotone.

(v) Let Ψ(x)Gc, where c is a constant and χ adG. Ψ is monotone,but not strongly pseudomonotone.

3. Auxiliary Problem Principle

In this section, we present the algorithm built on the auxiliary problemprinciple for solving variational inequalities involving general and single-valued operators. The version that is adapted to the multivalued case isstudied in Section 5.2.

3.1. Basic Algorithm. A whole family of algorithms to solve problem(1) has been unified in the framework of the so-called auxiliary problemprinciple (Refs. 18–21). We consider an auxiliary function M:χ→, whichis chosen to be differentiable and strongly convex, and a sequence of posi-tive numbers (k, k ∈ . For some x ∈ χ , we introduce the auxiliaryproblem

miny ∈ χ ad

(M(y)C⟨(kΨ(x)AM ′(x), y⟩). (6)

Let y(x) denote the solution of this problem. It is characterized by the vari-ational inequality

⟨M ′(y(x))C(kΨ(x)AM ′(x), yAy(x)⟩¤0, ∀ y ∈ χ ad. (7)

If y(x) happens to be equal to x, then it is checked easily that y(x) is also asolution of the original problem (1). This observation suggests the followingfixed-point algorithm.

Algorithm 3.1. Basic Algorithm.

Step 1. Start from some initial x0 in χ .


Step 2. At stage k, knowing xk, compute xkC1Gy(xk) by solving theauxiliary problem (6) with x set to xk,

minx ∈ χ ad

(M(x)C⟨(kΨ(xk)AM ′(xk), x⟩). (8)

Step 3. Stop if xkC1Axk is below some threshold. Otherwise, goback to (ii) with k←kC1.

The expression M( · )A⟨M ′(x), ·⟩ is sometimes referred to as a Bregmanfunction (Refs. 22–23).

Remark 3.1. At each step of this algorithm, xkC1 is the unique solu-tion of the variational problem

⟨Ψk(xkC1), xAxkC1⟩¤0, ∀ x ∈ χ ad,

where Ψk is the following approximation of Ψ:

Ψk(x)G(kΨ(xk)CM ′(x)AM ′(xk ).

The auxiliary function M may be chosen, for example, quadratic. Pang andChan (Ref. 24) call the algorithm where M ′(x)GG is a symmetric positive-definite matrix and (kG1, ∀ k ∈ , a projection algorithm. In particular, ifM( · )G · 22, then the solution of (8) is given explicitly by

xkC1GΠ(xkA(kΨ(xk)),

where Π denotes the projection onto χ ad. When Ψ is the gradient of someconvex function, this algorithm is the projected gradient algorithm.

The reader may find various linear approximations Ψk of Ψ in Ref. 1.

Remark 3.2. If χ is the product of N subspaces χ 1B· · ·BχN , and ifthe admissible set χ ad is decomposable, that is,

χ adGχ ad1 B· · ·Bχ ad

N , with χ adi ⊂ χ i for all i,

then choosing an additive auxiliary function

M(x)G ∑N

iG1

Mi (xi)

yields a decomposed auxiliary problem (8).

3.2. Review of Convergence Results. Under other technical assump-tions and appropriate choice of the stepsize (k in (8), the basic algorithmconverges when the operator Ψ is strongly monotone (Ref. 21) or has the


Dunn property (Ref. 16). The basic algorithm converges also when the oper-ator Ψ is a gradient and is monotone; in this case, problem (1) is equivalentto the minimization of a convex function (Refs. 18-19). However, it fails toconverge when the operator Ψ is simply monotone, but not a gradient. Weconsider the following counterexample given by the operator of rotation byπ2 in 2, which is represented by the matrix

0 −1

1 0.This linear operator is monotone, but not strongly monotone and does notsatisfy the Dunn property. By choosing M( · )G · 22 in (6), the sequencexkG(xk

1 , xk2) generated by the basic algorithm follows the recurrence

xkC11

xkC12

Gxk1

xk2A(k0 −1

1 0xk

1

xk2.

The norm of xk is always increasing for any positive (k; thus, it cannotconverge toward the unique solution x*G(0,0).

However, the Moreau–Yosida regularized operator of a monotoneoperator satisfies the Dunn property under which this algorithm converges.As a consequence, the regularized version of the basic algorithm converges,but unfortunately this is not easy to implement since the computation ofthe regularized operator is required at each stage of the regularization algo-rithm, which is an operation almost as difficult to achieve as to solve theoriginal problem. A way to circumvent this difficulty is proposed in Ref.25, where an explicit progressive regularization algorithm is presented.

Other convergence results based on a property called strong nestedmonotonicity, which is weaker than strong monotonicity and which doesnot even ensure monotonicity, are given by Cohen and Chaplais in Ref. 26.This property is weakened in Ref. 27, but the results are limited to the zero-finding problem of an affine operator in finite-dimensional spaces.

4. Convergence Theorem Based on the Pseudo-Dunn Property

Assumption 4.1.

(i) Ψ has the pseudo-Dunn property with constant E on χ ad.(ii) Ψ is uniform continuous on χ ad.(iii) M ′ is strongly monotone with constant b and Lipschitz con-

tinuous with constant B on χ ad.(iv) M ′ is continuous from χ equipped with the weak topology to χ

equipped with the weak topology.


Theorem 4.1. Suppose that problem (1) has a solution x*. If M ′ isstrongly monotone with constant b on χ ad, then there exists a unique solu-tion xkC1 to the auxiliary problem (8). Moreover, if Ψ has the pseudo-Dunnproperty with constant E on χ ad, and if

∀ k ∈ , (kC1⁄(k and αF(kF2b(ECβ), where αH0, βH0,

then the sequence xk is bounded, xkC1Axk converges to zero, and Ψ(xk)strongly converges toward Ψ(x*). In addition, if M ′ is Lipschitz continuouswith constant B, and if Ψ is uniform continuous on χ ad, then every weakcluster point of the sequence xk is a solution of (1). Moreover, if M ′ iscontinuous from χ equipped with the weak topology to χ equipped withthe weak topology, then the sequence xk weakly converges toward a solu-tion of problem (1).

Proof. The necessary and sufficient optimality condition satisfied byxkC1 is that

⟨M ′(xkC1)AM ′(xk), xAxkC1⟩C(k⟨Ψ(xk), xAxkC1⟩¤0, ∀ x ∈ χ ad . (9)

We consider the function Λ defined by

Λ(x, ()GΦ(x)CΩ(x, (),

where

Φ(x)GM(x*)AM(x)A⟨M ′(x), x*Ax⟩, (10a)

Ω(x, ()G(⟨Ψ(x*), xAx*⟩. (10b)

From the strong monotonicity of M ′,

Φ(xk)¤ (b2) xkAx*2¤0. (11)

Since Ω(xk, (k) is nonnegative,

Λ(xk, (k)¤ (b2) xkAx*2¤0. (12)

Let us study the variation of Λ for one stage of Algorithm 3.1,

ΓkC1k GΛ(xkC1, (kC1)AΛ(xk, (k).

We have that

ΓkC1k Gs1Cs2Cs3 ,

where

s1GM(xk)AM(xkC1)A⟨M ′(xk), xkAxkC1⟩,

s2G⟨M ′(xk)AM ′(xkC1), x*AxkC1⟩,

s3G(kC1⟨Ψ(x*), xkC1Ax*⟩A(k⟨Ψ(x*), xkAx*⟩.


By the strong monotonicity of M ′,

s1⁄A(b2) xkC1Axk2.

Using (9) with xGx* yields

s2 ⁄(k⟨Ψ(xk), x*AxkC1⟩

G(k⟨Ψ(xk), x*Axk⟩C(k⟨Ψ(xk), xkAxkC1⟩.

By using (1) with xGxk and the pseudo-Dunn assumption on Ψ, we get

⟨Ψ(xk), xkAx*⟩¤ (1E ) Ψ(xk)AΨ(x*) 2.

Thus,

s2⁄A((kE ) Ψ(xk)AΨ(x*) 2C(k⟨Ψ(xk), xkAxkC1⟩.

Since ∀ k, (kC1⁄(k, we obtain

s2Cs3⁄A((kE ) Ψ(xk)AΨ(x*) 2C(k⟨Ψ(xk)AΨ(x*), xkAxkC1⟩.

Therefore,

ΓkC1k ⁄A(b2) xkC1Axk2A((kE ) Ψ(xk)AΨ(x*) 2

C(kΨ(xk)AΨ(x*) xkC1Axk.

Then, by using the inequality

(kΨ(xk)AΨ(x*) xkC1Axk

⁄ (((k)22λ ) Ψ(xk)AΨ(x*) 2C(λ2) xkC1Axk2,

where λH0, we get

ΓkC1k ⁄A(b2Aλ2) xkC1Axk2

A(k(1EA(k2λ ) Ψ(xk)AΨ(x*) 2.

For

αF(kF2λ(ECβ),

where αH0 and βH0, we have

ΓkC1k ⁄A(b2Aλ2) xkC1Axk2

A(αβE(ECβ)) Ψ(xk)AΨ(x*) 2,

and for λFb, ΓkC1k is negative unless

xkC1Gxk and Ψ(xk)GΨ(x*).


Then, according to (9), xk is a solution to (1). The sequence Λ(xk, (k) isstrictly decreasing. But since it is positive, it must converge and the differ-ence between two consecutive terms tends to zero. Therefore, xkC1Axkand Ψ(xk)AΨ(x*) converge to zero. Moreover, since the sequenceΛ(xk, (k) converges, it is bounded, and so is also xk according to (12).

Let x be a weak cluster point of the sequence xk, and let xki be asubsequence weakly converging toward x. By using (9), since M ′ is Lipschitzcontinuous with constant B and (kHα , we have that, for some x ∈ χ ad,

⟨Ψ(xk), xAxkC1⟩¤A(Bα ) xkC1Axk xAxkC1.

Since xkiC1Axki converges to zero and since Ψ(xki ) strongly convergestoward Ψ(x*), then taking the limit for the subsequence ki in the lastinequality yields

⟨Ψ(x*), xAx⟩¤0, ∀ x ∈ χ ad. (13)

Moreover, we have that ⟨Ψ(x), xkiAx⟩ converges to 0. Either Ψ(x)G0, andthen x is a solution of (1), or Ψ(x)≠0. In this case, let

ykiGxkiA(⟨Ψ(x), xkiAx⟩Ψ(x) 2)Ψ(x).

Then,

⟨Ψ(x), ykiAx⟩G0. (14)

We have that ykiAxki converges to 0. Then, on the one hand, yki weaklyconverges toward x. On the other hand, since Ψ is uniform continuous,Ψ(yki ) strongly converges toward Ψ(x*).

Now, by using (14) and the pseudo-Dunn property of Ψ, we get

⟨Ψ(yki), ykiAx⟩¤ (1E ) Ψ(yki)AΨ(x) 2.

Therefore, by taking the limit for the subsequence ki in the last inequality,we have that Ψ(yki) strongly converges toward Ψ(x). Then, Ψ(x*)GΨ(x)and inequality (13) shows that x is a solution to problem (1).

Suppose now that M ′ is continuous from χ equipped with the weaktopology to χ equipped with the weak topology, and let us prove theuniqueness of x. Assume that the sequence xk has two weak cluster pointsx and x. Then, both cluster points can be used as x* to define the Lyapunovfunction Λ: this yields two possible Lyapunov functions, denoted Λ and Λrespectively. It was proved that Λ(xk, (k) has a limit that may depend onthe solution x* used to define Λ; then, the corresponding limits will bedenoted lr and l, respectively. Consider the subsequences ki and l j suchthat xki and xlj weakly converge toward x and x, respectively. Then, byusing (3) and the fact that Ψ(x)GΨ(x), we get

Λ(xki, (ki )GΛ(xki, (ki)CR(xki ),


where

R(xki )GM(x)AM(x)A⟨M ′(xki ), xAx⟩.

In the limit, Λ(xki, (ki ) and Λ(xki, (ki ) tend toward l and l, respectively. Sincethe subsequence xki converges weakly toward x, and since M ′ is continu-ous from χ equipped with the weak topology to χ equipped with the weaktopology, R(xki ) converges toward a limit l, such that

l¤ (b2) xAx2.

The latter inequality stems from the strong convexity of M. Therefore,

l¤ lrC(b2) xAx2.

By interchanging the role of x and x and of the subsequences ki andl j, the same calculations yield

lr¤ lC(b2) xAx2.

Then,

0⁄ (b2) xAx2⁄ lAlr,

0⁄ (b2) xAx2⁄ lrAl.

This proves that xGx.

5. Convergence Results Based on Strong Pseudomonotonicity

In this section, we prove the convergence of the auxiliary problemmethod for both the cases where the operator is single-valued and multival-ued, under the assumption that it is strongly pseudomonotone.

5.1. Case of a Single-Valued Operator.

Assumption 5.1.

(i) Ψ is strongly pseudomonotone with constant e on χ ad.(ii) Ψ is Lipschitz continuous with constant L on χ ad.(iii) M ′ is strongly monotone with constant b and Lipschitz continu-

ous with constant B on χ ad.

In this situation, all the assumptions on Ψ in Theorem 4.1 are satisfied;see the result of Lemma 2.2. Thus, we can state the following corollary ofTheorem 4.1.


Corollary 5.1. Suppose that problem (1) has a solution x*. If M ′ isstrongly monotone with constant b on χ ad, then there exists a unique solu-tion xkC1 to the auxiliary problem (8). If Ψ is strongly pseudomonotonewith constant e on χ ad (x* is then unique) and Lipschitz continuous withconstant L on χ ad, and if

∀ k ∈ , αF(kF2eb(L2Cβ), where αH0, βH0, (15)

then the sequence xk strongly converges toward x*. Moreover, if M ′ isLipschitz continuous with constant B on χ ad, then we have the a posteriorierror estimation

xkC1Ax*⁄ (Be(kCLe) xkC1Axk. (16)

Proof. Let us study the variation of the function Φ defined by (10a)for one stage of Algorithm 3.1,

∆kC1k GΦ(xkC1)AΦ(xk).

By using the same notation and similar calculations to those in the proofof Theorem 4.1, we get

∆kC1k Gs1Cs2 ,

with

s1⁄A(b2) xkC1Axk 2,

s2⁄(k⟨Ψ(xk), x*AxkC1⟩

G(k⟨Ψ(xk)AΨ(xkC1), x*AxkC1⟩C(k⟨Ψ(xkC1), x*AxkC1⟩.

By using (1) with xGxkC1 and the strong pseudomonotonicity of Ψ, we get

⟨Ψ(xkC1), xkC1Ax*⟩¤exkC1Ax*2. (17)

Thus,

s2⁄Ae(kxkC1Ax*2C(k⟨Ψ(xk)AΨ(xkC1), x*AxkC1⟩.

Therefore,

∆kC1k ⁄A(b2) xkC1Axk2Ae(kxkC1Ax*2

C(kΨ(xkC1)AΨ(xk) x*AxkC1

⁄A(b2) xkC1Axk2Ae(kxkC1Ax*2

C(kLxkC1Axk xkC1Ax*.


Then, by using the inequality

(kLxkC1Axk xkC1Ax*

⁄ (λ2) xkC1Axk2C(((k)2L22λ ) xkC1Ax*2,

with λGb, we get

∆kC1k ⁄ ((k)2(Ae(kCL22b) xkC1Ax*2

⁄ (Aα 2β2b) xkC1Ax*2.

The latter inequality follows from condition (15). Under this condition,∆kC1

k is negative unless xkC1Gx*. The sequence Φ(xk) is strictly decreas-ing. But since it is positive, it must converge and the difference between twoconsecutive terms tends to zero. Therefore, xk strongly converges towardx*.

Now, by using (9) with xGx* and (17), we have that

⟨M ′(xkC1)AM ′(xk), x*AxkC1⟩C(k⟨Ψ(xk)AΨ(xkC1), x*AxkC1⟩

¤e(kxkC1Ax*2.

Then, by using the Schwarz inequality in the first member and the Lipschitzassumptions on M ′ and Ψ, we get

BxkC1Axk xkC1Ax*C(kLxkC1Axk xkC1Ax*

¤e(kxkC1Ax*2.

We obtain inequality (16) after division by xkC1Ax*, which we assumenonzero; otherwise, the result is trivial.

5.2. Case of a Multivalued Operator. In this section, we consider themultivalued variational problem (2). The basic algorithm is modified as inRef. 21. Since Ψ(xk) is a set, we take any rk in this set to play the roleof Ψ(xk) in the case of a single-valued operator. Also, as for nonsmoothminimization problems (Refs. 20 and 28), the stepsizes (k satisfy the follow-ing conditions:

(kH0, ∑+S

kG0

(kG+S, ∑+S

kG0

((k)2F +S. (18)

Algorithm 5.1.

Step 1. Start from some initial x0 in χ .


Step 2. At stage k, knowing xk, compute xkC1 by solving the auxiliaryproblem

minx ∈ χad

(M(x)C⟨(krkAM ′(xk), x⟩), (19)

with rk ∈Ψ (xk).Step 3. Stop if xkC1Axk is below some threshold. Otherwise, go

back to (ii) with k←kC1.

Assumption 5.2.

(i) Ψ is strongly pseudomonotone with constant e on χ ad.(ii) Ψ satisfies

∃ αH0, ∃ βH0, such that,

∀ x ∈ χ ad, ∀ r ∈Ψ (x), r⁄α xCβ. (20)

(iii) M ′ is strongly monotone with constant b on χ ad.

The Lipschitz continuity assumption on Ψ in the single-valued case ischanged into assumption (20). It means that the norm of Ψ does notincrease faster than linearly with the norm of x.

The reader may find the next two lemmas in Ref. 20.

Lemma 5.1. Let xk and µk be two sequences in C such that

∑k ∈

µkF +S,

and with Xk denoting supl⁄k xl,

xN⁄ ∑NA1

kG1

µkXkC1CηN,

where ηN⁄η , ∀ N ∈ . Then, the sequence xk is bounded.

Lemma 5.2. Let f be a Lipschitz function on a Hilbert space χ andconsider the sequences xkk ∈ ⊂ χ and (kk ∈ ⊂ + such that

∑k ∈

(kG+S,

∃ ζ , ∀ k ∈ , xkC1Axk⁄ζ (k,

∃ ξ , ∑k ∈

(kf (xk)Aξ F +S.

Then,

limk→ +S

f (xk)Gξ .


Theorem 5.1. Assume that problem (2) has a solution x*. If M ′ isstrongly monotone with constant b on χ ad, then there exists a unique solu-tion xkC1 to the auxiliary problem (19). If Ψ is strongly pseudomonotonewith constant e on χ ad (x* is then unique), if it satisfies (20), and if thesequence (k satisfies (18), then the sequence xk strongly convergestoward xk.

Proof. The necessary and sufficient optimality condition satisfied byxkC1 is that

⟨M ′(xkC1)AM ′(xk), xAxkC1⟩C(k⟨rk, xAxkC1⟩¤0, ∀ x ∈ χ ad. (21)

Let us study the variation of the Lyapunov function Φ defined by (10a) forone stage of Algorithm 5.1. By using the same notation and similar calcu-lations to those in the proof of Corollary 5.1, we get

∆kC1k Gs1Cs2 ,

where

s1⁄A(b2) xkC1Axk2,

s2⁄(k⟨rk, x*AxkC1⟩

G(k⟨rk, x*Axk⟩C(k⟨rk, xkAxkC1⟩.

By using (2) with xGxk and the strong pseudomonotonicity of Ψ, we get

⟨rk, xkAx*⟩¤exkAx*2.

Thus,

s2⁄Ae(kxkAx*2C(k⟨rk, xkAxkC1⟩.

Therefore,

∆kC1k ⁄A(b2) xkC1Axk2Ae(kxkAx*2C(krk xkC1Axk.

By using the inequality

(krk xkC1Axk⁄ (((k)22λ ) rk2C(λ2) xkC1Axk2,

with λGb, we get

∆kC1k ⁄Ae(kxkAx*2C(((k)22b) rk2.

Using (20) yields

rk⁄α xkCβ

⁄α xkAx*Cα x*Cβ.


Then, by using the following inequality:

(uCû)2⁄2(u2Cû2),

with

uGα xkAx* and ûGα x*Cβ,

we get

rk22b⁄γ xkAx*2Cδ,

where

γGα 2b and δG(α x*Cβ)2b.

Thus,

∆kC1k ⁄Ae(kxkAx*2C((k)2(γ xkAx*2Cδ).

Now, summing up this inequality from kG0 to NA1 and using (11) yield,for all N,

(b2) xNAx*2⁄Φ(xN)

⁄Φ(x0)C ∑NA1

kG0Ae(kxkAx*2C((k)2γ xkAx*2Cδ

⁄Φ(x0)C ∑NA1

kG0

((k)2(γ xkAx*2Cδ). (22)

Then,

xNAx*2⁄ηNC ∑NA1

kG1

µkxkAx*2,

where

ηNG(2b)Φ(x0)C(2γ ((0)2b) x0Ax*2C ∑NA1

kG0

2δ((k)2b,

µkG2γ ((k)2b.

On the one hand, we have that

∀ k, xkAx*2⁄ supl⁄kC1

xlAx*2.

On the other hand, since

∑+S

kG0

((k)2F +S,


∃ η such that

ηN⁄η , ∀ N ∈ .

Then, by using Lemma 5.1, we conclude that xk is a bounded sequence.Therefore, on a bounded convex hull of this sequence, the functionx> xAx*2 is Lipschitz. Moreover, from (18), (22), and since thesequence xk is bounded, we get

∑k ∈

(kxkAx*2F +S.

Then, if we assume that xkC1Axk is nonnull, by using (21) with xGxk

and the strong monotonicity of M ′, we get

xkC1Axk⁄(krkb

⁄(k(α xkCβ)b;

but since xk is bounded, ∃ ζ such that

xkC1Axk⁄ζ (k.

This result is trivial when

xkC1AxkG0.

Finally, by using Lemma 5.2, we conclude that xk strongly converges towardx*.

6. Optimization Problems: Convergence Theorem Based on Pseudoconvexity

In this section, we assume that χ is finite-dimensional and we considerthe minimization problem on χ ad

minx ∈ χad

J(x). (23)

Definition 6.1. J is said to be coercive on χ ad if χ ad is bounded or

limx→+Sx ∈ χad

J(x)G+S.

Definition 6.2. A differentiable function J on an open subset C of χis pseudoconvex on C if

∀ x1 , x2 ∈ C, ⟨J′(x1), x2Ax1 ⟩¤0 ⇒ J(x2)¤J(x1).


Theorem 6.1. See Ref. 2. Let J be differentiable on an open convexsubset C of χ . Then, J is pseudoconvex if and only if the gradient J′(x) ispseudomonotone on C.

Definition 6.3. A function J on an open subset C of χ is quasiconvexif

∀ x1 , x2 ∈ C, ∀ λ ∈ [0, 1], J(λx1C(1Aλ )x2)⁄max(J(x1), J(x2)),

which is equivalent to

∀ x1 , x2 ∈ C, ∀ λ ∈ [0, 1], J(x1)⁄J(x2) ⇒ J(λx1C(1Aλ )x2)⁄J(x2).

Geometrically, the quasiconvexity is characterized by the following level-setproperty:

x ∈ CJ(x)⁄α is convex for all α ∈ .

For differentiable functions, we have the following results.

Theorem 6.2. See Refs. 29 and 9. A differentiable function J is quasi-convex on an open convex subset C if and only if

∀ x1 , x2 ∈ C, J(x1)⁄J(x2) ⇒ ⟨ J′(x2), x1Ax2 ⟩⁄0.

Theorem 6.3. See Ref. 3. Let J be differentiable on an open convexsubset C of χ . Then, J is quasiconvex if and only if the gradient J′(x) isquasimonotone on C.

Lemma 6.1. If J is differentiable and pseudoconvex on an open con-vex subset that contains χ ad, then x* is a solution of (23) if and only if x*is a solution of the variational inequality

⟨J′(x*), xAx*⟩¤0, ∀ x ∈ χ ad. (24)

Remark 6.1. The result of Lemma 6.1 is not true when J is quasicon-vex. This is shown in the following example.

Example 6.1. Let χ adG[0, a], where aH0 and J(x)GAx22. J isquasiconvex but not pseudoconvex. The minimization problem (23) has aunique solution x*Ga, but both x*Ga and y*G0 are solutions of (24).

In the rest of this section, we assume that J is differentiable and pseudo-convex on an open convex subset C that contains χ ad.

We consider the algorithm built on the auxiliary problem principlewhere the auxiliary function and the stepsize may vary with the iteration


index k, as presented in Ref. 19. Let a sequence of differentiable andstrongly convex functions Mk, k ∈ and positive numbers (k, k ∈ bechosen.

Algorithm 6.1.

Step 1. Start from some initial x0 in χ .Step 2. At stage k, knowing xk, compute xkC1 by solving the auxiliary

problem

minx ∈ χad

(Mk(x)C⟨(kJ′(xk)A(Mk)′(xk), x⟩). (25)

Step 3. Stop if xkC1Axk is below some threshold. Otherwise, goback to (ii) with k←kC1.

Assumption 6.1.

(i) J is coercive, lower semicontinuous on χ ad, and there existsu ∈ χ ad where J is finite.

(ii) J′ is Lipschitz continuous with constant L on χ ad.(iii) J is pseudoconvex on C.(iv) (Mk)′ is strongly monotone with constant bk and Lipschitz con-

tinuous with constant Bk on χ ad. Moreover, ∃ bH0 and BH0 suchthat, ∀ k ∈ , bk„b and B k‚B.

Theorem 6.4. Assume that J is coercive, lower semicontinuous on χ ad,and that there exists u ∈ χ ad where J is finite; then, there exists a solution x*to (23). In addition, if (Mk)′ is strongly monotone with constant bk, withbk„bH0, ∀ k ∈ , on χ ad, then there exists a unique solution xkC1 to theauxiliary problem (25).

Moreover, if J′ is Lipschitz continuous with constant L on χ ad, and ifthe (k are such that

∀ k ∈ , αF(kF2bk(LCβ), where αH0, βH0, (26)

then the sequence J(xk) is strictly decreasing, unless xk is a solution x* to(23) for some k; the sequence xk is bounded and xkC1Axk converges tozero.

In addition, if we assume that (Mk)′ is Lipschitz continuous with con-stant Bk on χ ad, that ∃ BH0 such that, ∀ k ∈ , B k‚B, and that J is pseudo-convex on C, then every cluster point of the sequence xk is a solution ofproblem (23).

The proof of this theorem can be found in Ref. 30.


Remark 6.2. If χ is not supposed to be finite-dimensional in Theorem6.4, then in order to have the optimality condition (24), it seems to us neces-sary to assume that the gradient J′(x) is weakly continuous, which is astrong condition. This is the reason why the convergence results in Theorem6.4 are limited to the finite-dimensional case.

Remark 6.3. If, ∀ k ∈ , (kG( in Algorithm 3.1 [respectively, Algo-rithm 6.1], then the condition on ( in Theorem 4.1. and Corollary 5.1[respectively, Theorem 6.4] may be simplified by setting αGβG0.

7. Conclusions

In this paper, we proved the convergence of the auxiliary problemmethod in Hilbert spaces, when the operator involved in the variationalinequality problem has the pseudo. Dunn property or is strongly pseudo-monotone. We also proved the convergence in finite-dimension when theoperator is the gradient of a pseudoconvex function. The counterexampleof the operator of rotation by π2 shows that this cannot be extended togeneral pseudomonotone and even monotone variational inequalities. InRef. 31, we show how the Moreau–Yosida regularization and the progress-ive regularization method (developed in Ref. 25) can still be extended topseudomonotone variational inequalities involving general operators.

Acknowledgment

The author is greatly indebted to Professor Guy Cohen for his carefulreading and his precious advice.

References

1. HARKER, P. T., and PANG, J. S., Finite-Dimensional Variational Inequality andNonlinear Complementarity Problems: A Surûey of Theory, Algorithms, andApplications, Mathematical Programming, Vol. 48, pp. 161–220, 1990.

2. KARAMARDIAN, S., Complementarity Problems oûer Cones with Monotone andPseudomonotone Maps, Journal of Optimization Theory and Applications, Vol.18, pp. 445–455, 1976.

3. KARAMARDIAN, S., and SCHAIBLE, S., Seûen Kinds of Monotone Maps, Journalof Optimization Theory and Applications, Vol. 66, pp. 37–47, 1990.


4. SCHAIBLE, S., Generalized Monotonicity: A Surûey, Generalized Convexity,Edited by S. Komlosi, R. Rapcsak, and S. Schaible, Springer Verlag, Heidel-berg, Germany, pp. 229–249, 1994.

5. KARAMARDIAN, S., SCHAIBLE, S., and CROUZEIX, J. P., Characterizations ofGeneralized Monotone Maps, Journal of Optimization Theory and Applications,Vol. 76, pp. 399–413, 1993.

6. KOMLOSI, S., Generalized Monotonicity and Generalized Conûexity, Journal ofOptimization Theory and Applications, Vol. 84, pp. 361–376, 1995.

7. ZHU, D., and MARCOTTE, P., New Classes of Generalized Monotonicity, Journalof Optimization Theory and Applications, Vol. 87, pp. 457–471, 1995.

8. MANGASARIAN, O. L., Pseudoconûex Functions, SIAM Journal on Control, Vol.3, pp. 281–290, 1965.

9. MANGASARIAN, O. L., Nonlinear Programming, SIAM, Philadelphia, Pennsyl-vania, 1994.

10. SCHAIBLE, S., Generalized Monotonicity: Concepts and Uses, VariationalInequalities and Network Equilibrium Problems, Edited by F. Giannessi and A.Maugeri, Plenum Publishing Corporation, New York, NY, pp. 289–299, 1995.

11. YAO, J. C., Variational Inequalities with Generalized Monotone Operators, Math-ematics of Operations Research, Vol. 19, pp. 691–705, 1994.

12. YAO, J. C., Multiûalued Variational Inequalities with K-Pseudomonotone Oper-ators, Journal of Optimization Theory and Applications, Vol. 83, pp. 391–403,1994.

13. CROUZEIX, J. P., Pseudomonotone Variational Inequality Problems: Existence ofSolutions, Mathematical Programming, Vol. 78, pp. 305–314, 1997.

14. ZHU, D. L., and MARCOTTE, P., Cocoerciûity and Its Role in the Conûergenceof Iteratiûe Schemes for Solûing Variational Inequalities, SIAM Journal on Opti-mization, Vol. 6, pp. 714–726, 1996.

15. ECKSTEIN, J., and BERTSEKAS, D. P., On the Douglas–Rachford SplittingMethod and the Proximal Point Algorithm for Maximal Monotone Operators,Mathematical Programming, Vol. 55, pp. 293–318, 1992.

16. MATAOUI, M. A., Contributions a la Decomposition et a l ’Agregation des Pro-blemes Variationnels, Thesis Dissertation, Ecole des Mines de Paris, Fontaineb-leau, France, 1990.

17. BREZIS, H., Equations et Inequations Nonlineaires dans les Espaces Vectoriels enDualite, Annales de l’Institut Fourier, Vol. 18, pp. 115–175, 1968.

18. COHEN, G., Optimization by Decomposition and Coordination: A UnifiedApproach, IEEE Transactions on Automatic Control, Vol. 23, pp. 222–232,1978.

19. COHEN, G., Auxiliary Problem Principle and Decomposition of OptimizationProblems, Journal of Optimization Theory and Applications, Vol. 32, pp. 277–305, 1980.

20. COHEN, G., and ZHU, D. L., Decomposition-Coordination Methods in LargeScale Optimization Problems: The Nondifferentiable Case and the Use of Aug-mented Lagrangians, Advances in Large Scale Systems Theory and Applications,Edited by J. B. Cruz, JAI Press, Greenwich, Connecticut, Vol. 1, pp. 203–266,1983.


21. COHEN, G., Auxiliary Problem Principle Extended to Variational Inequalities,Journal of Optimization Theory and Applications, Vol. 59, pp. 325–333, 1988.

22. BREGMAN, L. M., The Relaxation Method of Finding the Common Point of Con-ûex Sets and Its Applications to the Solution of Problems in Conûex Programming,USSR Computational Mathematics and Mathematical Physics, Vol. 7, pp. 200–217, 1967.

23. ECKSTEIN, J., Nonlinear Proximal Point Algorithms Using Bregman Functions,Mathematics of Operations Research, Vol. 18, pp. 202–226, 1993.

24. PANG, J. S., and CHAN, D., Iteratiûe Methods for Variational and Comp-lementarity Problems, Mathematical Programming, Vol. 24, pp. 284–313, 1982.

25. EL FAROUQ, N., and COHEN, G., Progressiûe Regularization of VariationalInequalities and Decomposition Algorithms, Journal of Optimization Theory andApplications, Vol. 97, pp. 407–433, 1998.

26. COHEN, G., and CHAPLAIS, F., Nested Monotony for Variational Inequalitiesoûer Product of Spaces and Conûergence of Iteratiûe Algorithms, Journal of Opti-mization Theory and Applications, Vol. 59, pp. 369–390, 1988.

27. EL FAROUQ, N., and COHEN, G., Conûergence of a Zero-Finding Algorithm forSome Nonmonotone Linear Operators, Preprint, Ecole des Mines, Paris, France,1998.

28. AUSLENDER, A., Optimization, Methodes Numeriques, Masson, Paris, France,1976.

29. ARROW, K. J., and ENTHOVEN, A. C., Quasiconcaûe Programming, Econome-tria, Vol. 29, pp. 779–800, 1961.

30. EL FAROUQ, N., Pseudoconûex Optimization Problems: Conûergence of theAuxiliary Problem Method, Report 00178, LAAS CNRS, Toulouse, France,2000.

31. EL FAROUQ, N., A Conûergent Algorithm Based on the Progressiûe Regulariz-ation for Solûing Pseudomonotone Variational Inequalities, Preprint, LAASCNRS, Toulouse, France, 2000.

Pseudomonotone Variational Inequalities: Convergence of the …naelfaro/publis/ElFarouq-JOTA... · 2021. 3. 31. · N. EL FAROUQ 1 Communicated by Z. Q. Luo Abstract. This paper deals

Documents