Research Article A New Conjugate Gradient Algorithm with …downloads.hindawi.com/journals/mpe/2015/352524.pdf · 2019-07-31 · Research Article A New Conjugate Gradient Algorithm

Research ArticleA New Conjugate Gradient Algorithm with Sufficient DescentProperty for Unconstrained Optimization

XiaoPing Wu1 LiYing Liu2 FengJie Xie1 and YongFei Li1

1School of Economic Management Xirsquoan University of Posts and Telecommunications Shaanxi Xirsquoan 710061 China2School of Mathematics Science Liaocheng University Shandong Liaocheng 252000 China

Correspondence should be addressed to XiaoPing Wu wuxiaoping1978126com

Received 16 May 2015 Revised 24 September 2015 Accepted 29 September 2015

Academic Editor Masoud Hajarian

Copyright copy 2015 XiaoPing Wu et al This is an open access article distributed under the Creative Commons Attribution Licensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

A new nonlinear conjugate gradient formula which satisfies the sufficient descent condition for solving unconstrainedoptimization problem is proposed The global convergence of the algorithm is established under weak Wolfe line search Somenumerical experiments show that this new WWPNPRP+ algorithm is competitive to the SWPPRP+ algorithm the SWPHS+algorithm and the WWPDYHS+ algorithm

1 Introduction

In this paper we consider the following unconstrained opti-mization problem

min 119891 (119909) | 119909 isinR119899 (1)

where 119891(119909) R119899 rarr R is a twice continuously differentiablefunction whose gradient is denoted by 119892(119909) R119899 rarr R119899 Itsiterative formula is given by

119909119896+1= 119909119896+ 119905119896119889119896 (2)

where

119889119896=

minus119892119896

if 119896 = 1

minus119892119896+ 120573119896119889119896minus1

if 119896 ge 2(3)

and 119905119896is a step size which is computed by carrying out a line

search 120573119896is a scalar and 119892

119896denotes 119892(119909

119896) There are at least

six famous formulas for 120573119896 which are given below

120573HS119896=119892119879119896(119892119896minus 119892119896minus1)

(119892119896minus 119892119896minus1)119879

119889119896minus1

120573FR119896=119892119879119896119892119896

119892119879119896minus1119892119896minus1

120573PRP119896=119892119879119896(119892119896minus 119892119896minus1)

119892119879119896minus1119892119896minus1

120573CD119896= minus

119892119879119896119892119896

119889119879119896minus1119892119896minus1

120573LS119896= minus119892119879119896(119892119896minus 119892119896minus1)

119889119879119896minus1119892119896minus1

120573DY119896=

119892119879119896119892119896

(119892119896minus 119892119896minus1)119879

119889119896minus1

(4)

To establish the global convergence results of the aboveconjugate gradient (CG) methods it is usually required thatthe step size 119905

119896should satisfy some line search conditions

such as the weak Wolfe-Powell (WWP) line search

119891 (119909119896+ 119905119896119889119896) minus 119891 (119909

119896) le 120575119905

119896119892119879119896119889119896 (5)

119892 (119909119896+ 119905119896119889119896)119879

119889119896ge 120590119892119879119896119889119896 (6)

where 120575 isin (0 12) and 120590 isin (120575 1) and strong Wolfe-Powell(SWP) line search (5) and

100381610038161003816100381610038161003816119892 (119909119896+ 119905119896119889119896)119879

119889119896

100381610038161003816100381610038161003816le minus120590119892119879

119896119889119896 (7)

Hindawi Publishing CorporationMathematical Problems in EngineeringVolume 2015 Article ID 352524 8 pageshttpdxdoiorg1011552015352524

2 Mathematical Problems in Engineering

where 120575 isin (0 12) and 120590 isin (120575 1) Wolfe-Powell is referred toas Wolfe

Considerable attentions have been made on the globalconvergence behaviors for the above methods Zoutendijk[1] proved that the FR method with exact line search isglobally convergent Al-Baali [2] extended this result to thestrong Wolfe line search conditions In [3] Dai and Yuan

proposed the DY method which produces a descent searchdirection at every iteration and converges globally providedthat the line search satisfies the weak Wolfe conditions In[4] Wei et al discussed the global convergence of the PRPconjugate gradient method (CGM) with inexact line searchfor nonconvex unconstrained optimization Recently basedon [5ndash7] Jiang et al [8] proposed a hybrid CGM with

120573JHJ119896=

100381710038171003817100381711989211989610038171003817100381710038172

minusmax 0 (10038171003817100381710038171198921198961003817100381710038171003817 1003817100381710038171003817119889119896minus1

1003817100381710038171003817) 119892119879

119896119889119896minus1 (1003817100381710038171003817119892119896

1003817100381710038171003817 1003817100381710038171003817119892119896minus1

1003817100381710038171003817) 119892119879

119896119892119896minus1

119889119879119896minus1(119892119896minus 119892119896minus1)

(8)

Under the Wolfe line search the method possesses globalconvergence and efficient numerical performance

On some studies of the conjugate gradient methods thesufficient descent condition

119892119879119896119889119896le minus119888 1003817100381710038171003817119892119896

10038171003817100381710038172

119888 gt 0 (9)

is often used to analyze the global convergence of thenonlinear conjugate gradient method with the inexact linesearch techniques For instance Touati-Ahmed and Storey[9] Al-Baali [2] Gilbert and Nocedal [10] and Hu andStorey [11] hinted that the sufficient descent condition maybe crucial for conjugate gradientmethods Unfortunately thiscondition is hard to hold It has been showed that the PRPmethod with the strong Wolfe Powell line search does notensure this condition at each iteration So Grippo and Lucidi[12] managed to find some line searches which ensure thesufficient descent condition and they presented a new linesearch which ensures this condition The convergence of thePRP method with this line search had been established Yuet al [13] analyzed the global convergence of modified PRPCGM with sufficient descent property Gilbert and Nocedal[10] gave another way to discuss the global convergence ofthe PRP method with the weak Wolfe line search By using acomplicated line search they were able to establish the globalconvergence result of the PRP and HSmethods by restrictingthe parameter 120573

119896in (3) not allowed to be negative that is

120573+119896= max 0 120573PRP

119896 (10)

which yields a globally convergent CG method being alsocomputationally efficient [14] In spite of the numericalefficiency of the PRP method as an important defect themethod lacks the following descent property

119892119879119896119889119896le 0 forall119896 ge 0 (11)

even for uniformly convex objective functions [15] Thismotivated the researchers to pay much attention to findingsome extensions of the PRPmethodwith descent property Inthis context Yu et al [16] proposed a modified form of 120573PRP

119896

as follows

120573DPRP119896

= 120573PRP119896minus 119862100381710038171003817100381711991011989610038171003817100381710038172

100381710038171003817100381711989211989610038171003817100381710038174119892119879119896+1119889119896 (12)

with a constant 119862 ge 14 leading to a CG method withthe sufficient descent property Dai and Kou [17] proposea family of conjugate gradient methods and an improvedWolfe line search meanwhile to accelerate the algorithman adaptive restart along negative gradients method is intro-duced Jiang and Jian [18] proposed twomodified CGMswithdisturbance factors based on a variant of PRP method thetwo proposed methods not only generate sufficient descentdirection at each iteration but also converge globally fornonconvex minimization if the strong Wolfe line search isused A newhybrid conjugate gradientmethodwas presentedfor unconstrained optimization The proposed method cangenerate decent directions at every iteration moreover thisproperty is independent of the steplength line search Underthe Wolfe line search the proposed method possesses globalconvergence [19]

The main purpose of this paper is to design an efficientalgorithm which possesses the properties of global conver-gence sufficient descent and good numerical results In nextsection we present a new CG formula and give its propertiesIn Section 3 the new algorithm and its global convergenceresult will be established To test and compare the numericalperformance of the proposed method in the last part of thiswork a large amount of medium-scale numerical experi-ments are reported by tables and performance profiles

2 The Formula and Its Property

Because sufficient descent condition (9) is a very nice andimportant property to analyze the global convergence of theCG methods we hope to find 120573

119896such that 119889

119896satisfies (9) In

the following we propose a sequence 120573119896 and prove that it

has such property Firstly we give a definition of a descentsequence (or a sufficient descent sequence) a sequence 120573

119896

is called a descent sequence (or a sufficient descent sequence)for the CG methods if there exists a constant 120591 isin (0 1) (or120591 isin [0 1)) such that for all 119896 ge 2

120573119896119892119879119896119889119896minus1le 120591 1003817100381710038171003817119892119896

10038171003817100381710038172

(13)

By using (3) we have for all 119896 ge 2

119892119879119896119889119896= minus 1003817100381710038171003817119892119896

10038171003817100381710038172

+ 120573119896119892119879119896119889119896minus1 (14)

Mathematical Problems in Engineering 3

From the above discussion we require that

minus 100381710038171003817100381711989211989610038171003817100381710038172

+ 120573119896119892119879119896119889119896minus1le 0 (15)

The above inequality implies (13)In [20] the authors proposed a variation of the FR

formula

120573VFR119896(120583) =

1205831

100381710038171003817100381711989211989610038171003817100381710038172

1205832

1003816100381610038161003816119892119879

119896119889119896minus1

1003816100381610038161003816 + 12058331003817100381710038171003817119892119896minus1

10038171003817100381710038172 (16)

where 1205831isin (0 +infin) 120583

2isin [1205831+ 1205981 +infin) 120583

3isin (0 +infin) and 120598

1

is any given positive constant It is easy to prove that 120573VFR119896 is

a descent sequence (with 120591 = 12058311205832) for CGmethds if 119892119879

119896119889119896le

0 Formula (16) possesses the sufficient descent property andproved that there exist some nonlinear conjugate gradientformulae possessing the sufficient descent property withoutany line searches where

120573WYL119896=119892119879119896(119892119896minus 1003817100381710038171003817119892119896

1003817100381710038171003817 1003817100381710038171003817119892119896 minus 1

1003817100381710038171003817)

119892119879119896minus1119892119896minus1

(17)

By restricting the parameter 120590 le 14 under the SWP linesearch condition the WYL method possessed the sufficientdescent condition [21]

In [22] the authors designed the following variation of thePRP formula which possesses the sufficient descent propertywithout any line searches

120573119873119896(120583)

=

100381710038171003817100381711989211989610038171003817100381710038172

minus10038161003816100381610038161003816119892119879

119896119889119896minus1

10038161003816100381610038161003816120583 1003816100381610038161003816119892119879

119896119889119896minus1

1003816100381610038161003816 +1003817100381710038171003817119892119896minus1

10038171003817100381710038172

if 100381710038171003817100381711989211989610038171003817100381710038172

ge10038161003816100381610038161003816119892119879

119896119889119896minus1

10038161003816100381610038161003816

0 Otherwise

(18)

in which 120583 ge 1Motivated by the ideas in [20 22] without any line

search and sufficient descent and taking into account thegood convergence properties of [10] and the good numericalperformance in [14] we propose a class new formula about120573119896as follows

120573NPRP119896

=1205821205831

100381710038171003817100381711989211989610038171003817100381710038172

+ (1 minus 120582) 1205831(100381710038171003817100381711989211989610038171003817100381710038172

minus10038161003816100381610038161003816119892119879

119896119892119896minus1

10038161003816100381610038161003816)

1205832

1003816100381610038161003816119892119879

119896119889119896minus1

1003816100381610038161003816 + 12058311003817100381710038171003817119892119896minus1

10038171003817100381710038172

(19)

where the definitions of 1205831 1205832are the same as those in

formula (16) 120582 isin (0 1)In order to ensure the nonnegative of the parameter 120573

119896

we define

120573NPRP+

119896= max 0 120573NPRP

119896 (20)

Thus if a negative of120573NPRP119896

occurs this strategy will restart theiteration along the steepest direction

The following two propositions show that the 120573NPRP+

119896 is

a descent sequence so that 119889119896can make sufficient descent

condition (9) hold

Proposition 1 Suppose that 120573119896is defined by (19)-(20) then

one has that

120573NPRP+

119896le

100381710038171003817100381711989211989610038171003817100381710038172

1003816100381610038161003816119892119879

119896119889119896minus1

1003816100381610038161003816 (21)

where 0 lt = 12058311205832lt 1

Proof It is clear that inequality (21) holds when 120573NPRP+

119896= 0

Now we consider the case where 120573NPRP+

119896= 120573NPRP119896

So we have

120573NPRP+

119896=1205821205831

100381710038171003817100381711989211989610038171003817100381710038172

+ (1 minus 120582) 1205831(100381710038171003817100381711989211989610038171003817100381710038172

minus10038161003816100381610038161003816119892119879

119896119892119896minus1

10038161003816100381610038161003816)

1205832

1003816100381610038161003816119892119879

119896119889119896minus1

1003816100381610038161003816 + 12058311003817100381710038171003817119892119896minus1

10038171003817100381710038172

le1205821205831

100381710038171003817100381710038171198922

119896

10038171003817100381710038171003817 + (1 minus 120582) 1205831100381710038171003817100381711989211989610038171003817100381710038172

1205832

1003816100381610038161003816119892119879

119896119889119896minus1

1003816100381610038161003816le1205831

1205832

100381710038171003817100381711989211989610038171003817100381710038172

1003816100381610038161003816119892119879

119896119889119896minus1

1003816100381610038161003816

= 100381710038171003817100381711989211989610038171003817100381710038172

1003816100381610038161003816119892119879

119896119889119896minus1

1003816100381610038161003816

(22)

Hence 120573NPRP+

119896can make (21) hold Furthermore 120573NPRP

+

119896 is a

descent sequence without any line search

Proposition 2 Suppose that120573119896is defined by (19)-(20) then119889

119896

satisfies the sufficient descent condition (9) for all 119896 ge 1 where119888 = 120582(1 minus 120583

11205832)

Proof For any 119896 gt 1 suppose that 119892119879119896minus1119889119896minus1lt 0

If 120573NPRP+

119896= 0 then 119889

119896= minus119892119896 So we have

119892119879119896119889119896= minus 1003817100381710038171003817119892119896

10038171003817100381710038172

le minus119888 100381710038171003817100381711989211989610038171003817100381710038172

(23)

where 119888 = 120582(1 minus 12058311205832)

Otherwise from the definition of 120573NPRP+

119896 we can obtain

119892119879119896119889119896= 119892119879119896

[

[

minus119892119896

+ (1205821205831

100381710038171003817100381711989211989610038171003817100381710038172

+ (1 minus 120582) 1205831(100381710038171003817100381711989211989610038171003817100381710038172

minus10038161003816100381610038161003816119892119879

119896119892119896minus1

10038161003816100381610038161003816)

1205832

1003816100381610038161003816119892119879

119896119889119896minus1

1003816100381610038161003816 + 12058311003817100381710038171003817119892119896minus1

10038171003817100381710038172

)119889119896minus1

]

]

le minus 100381710038171003817100381711989211989610038171003817100381710038172

+ (1205821205831

100381710038171003817100381711989211989610038171003817100381710038172

1205832

10038161003816100381610038161003816119892119879

119896minus1119889119896minus1

10038161003816100381610038161003816

10038161003816100381610038161003816119892119879

119896119889119896minus1

10038161003816100381610038161003816

+(1 minus 120582) 120583

1

100381710038171003817100381711989211989610038171003817100381710038172

1205832

1003816100381610038161003816119892119879

119896119889119896minus1

1003816100381610038161003816

10038161003816100381610038161003816119892119879

119896119889119896minus1

10038161003816100381610038161003816) le minus100381710038171003817100381711989211989610038171003817100381710038172

+ 120582

sdot1205831

1205832

100381710038171003817100381711989211989610038171003817100381710038172

+ 100381710038171003817100381711989211989610038171003817100381710038172

minus 120582 100381710038171003817100381711989211989610038171003817100381710038172

le minus120582 100381710038171003817100381711989211989610038171003817100381710038172

+ 120582

sdot1205831

1205832

100381710038171003817100381711989211989610038171003817100381710038172

le minus120582(1 minus1205831

1205832

) 100381710038171003817100381711989211989610038171003817100381710038172

= minus119888 100381710038171003817100381711989211989610038171003817100381710038172

(24)

For 11989211987911198891= minus119892

12 lt 0 we can deduce that 119889

119896can make

sufficient descent condition (9) hold for all 119896 ge 1

By the proof of Proposition 2 we can know that theformula 120583

11205832lt 1 is necessary otherwise the sufficient

descent condition can not be held

4 Mathematical Problems in Engineering

3 Global Convergence

In this section we propose an algorithm related to 120573NPRP+

119896

and then we study the global convergence property of thisalgorithm Firstly we make the following two assumptionswhich have been widely used in the literature to analyze theglobal convergence of the CG methods with the inexact linesearches

Assumption A The level set

Ω = 119909 isin 119877119899 | 119891 (119909) le 119891 (1199091) (25)

is bounded

Assumption B The gradient 119892(119909) is Lipschitz continuousthat is there exists a constant 119871 gt 0 such that for any 119909 119910 isinΩ

1003817100381710038171003817119892 (119909) minus 119892 (119910)1003817100381710038171003817 le 119871

1003817100381710038171003817119909 minus 1199101003817100381710038171003817 (26)

Now we give the algorithm

Algorithm 3

Step 0 Given 1199091isin 119877119899 set 119889

1= minus1198921 119896 = 1 If 119892

1= 0 then

stop Otherwise go to Step 1

Step 1 Find 119905119896gt 0 satisfying weak Wolfe conditions (5) and

(6)

Step 2 Let 119909119896+1= 119909119896+ 119905119896119889119896and 119892

119896+1= 119892(119909

119896+1) If 119892

119896+1 = 0

then stop Otherwise go to Step 3

Step 3 Compute 120573119873119875119877119875+

119896+1by formula (19) and (20) Then

generate 119889119896+1

by (3)

Step 4 Set 119896 = 119896 + 1 go to Step 0

Since 119891(119909119896) is decreasing sequence it is clear that the

sequence 119909119896 is contained in Ω and there exists a constant

119891lowast such that

lim119896rarrinfin

119891 (119909119896) = 119891lowast (27)

By using Assumptions A and B we can deduce that thereexists119872 gt 0 such that

10038171003817100381710038171198921198961003817100381710038171003817 le 119872 forall119909 isin Ω (28)

The following important result was obtained by Zoutendijk[1] and Wolfe [23 24]

Lemma 4 Suppose 119891(119909) is bounded below and 119892(119909) satisfiesthe Lipschitz condition Consider any iteration method offormula (2) where 119889

119896satisfies 119889119879

119896119892119896lt 0 and 119905

119896is obtained

by the weak Wolf line search Then

infin

sum119896=1

(119892119879119896119889119896)2

100381710038171003817100381711988911989610038171003817100381710038172lt +infin (29)

The following lemma was obtained by Dai and Yuan [25]

Lemma 5 Assume that a positive series 119886119894 satisfies the fol-

lowing inequality for all 119896

119896

sum119894=1

119886119894ge 119897119896 + 119888 (30)

where 119897 gt 0 and 119888 are constant Then one has

sum119894ge1

1198862119894

119894= +infin

sum119896ge1

1198862119896

sum119896119894=1119886119894

= +infin

(31)

Theorem 6 Suppose that Assumptions A and B hold 119909119896 is a

sequence generated by Algorithm 3 Then one has

lim inf119896rarrinfin

10038171003817100381710038171198921198961003817100381710038171003817 = 0 (32)

Proof Equation (3) indicates that for all 119896 ge 2

119889119896+ 119892119896= 120573119896119889119896minus1 (33)

Squaring both sides of (33) we obtain

100381710038171003817100381711988911989610038171003817100381710038172

= minus 100381710038171003817100381711989211989610038171003817100381710038172

minus 2119892119879119896119889119896+ 1205732119896

1003817100381710038171003817119889119896minus110038171003817100381710038172

(34)

Suppose that 120573NPRP+

119896= 120573NPRP119896

in (20) Then

100381710038171003817100381711988911989610038171003817100381710038172

= minus 100381710038171003817100381711989211989610038171003817100381710038172

minus 2119892119879119896119889119896+ (120573NPRP

+

119896)2 1003817100381710038171003817119889119896minus1

10038171003817100381710038172

= minus 100381710038171003817100381711989211989610038171003817100381710038172

minus 2119892119879119896119889119896

+ (1205821205831

100381710038171003817100381711989211989610038171003817100381710038172

+ (1 minus 120582) 1205831(100381710038171003817100381711989211989610038171003817100381710038172

minus10038161003816100381610038161003816119892119879

119896119892119896minus1

10038161003816100381610038161003816)

1205832

1003816100381610038161003816119892119879

119896119889119896minus1

1003816100381610038161003816 + 12058311003817100381710038171003817119892119896minus1

10038171003817100381710038172

)

2

sdot 1003817100381710038171003817119889119896minus110038171003817100381710038172

le minus 100381710038171003817100381711989211989610038171003817100381710038172

minus 2119892119879119896119889119896+ 1003817100381710038171003817119892119896

10038171003817100381710038174

1003817100381710038171003817119889119896minus110038171003817100381710038172

1003817100381710038171003817119892119896minus110038171003817100381710038174

(35)

We have

ℎ119896le ℎ119896minus1minus1100381710038171003817100381711989211989610038171003817100381710038172+2120574119896

100381710038171003817100381711989211989610038171003817100381710038172 (36)

where ℎ119896= 11988911989621198921198964 and 120574

119896= minus1198921198791198961198891198961198921198962

Note that ℎ1= 1119892

12 and 120574

1= 1 It follows from (36)

that

ℎ119896le minus119896

sum119894=1

1100381710038171003817100381711989211989410038171003817100381710038172+ 2119896

sum119894=1

10038161003816100381610038161205741198941003816100381610038161003816

100381710038171003817100381711989211989410038171003817100381710038172 (37)

Suppose that conclusion (32) does not holdThen there existsa positive scalar 120598 such that for all 119896 ge 1

10038171003817100381710038171198921198961003817100381710038171003817 ge 120598 (38)

Mathematical Problems in Engineering 5

Thus it follows from (28) and (38) that

ℎ119896le minus119896

1198722+2

1205982

119896

sum119894=1

10038161003816100381610038161205741198941003816100381610038161003816 (39)

Further we have

ℎ119896le2

1205982

119896

sum119894=1

10038161003816100381610038161205741198941003816100381610038161003816 (40)

On the other hand using ℎ119896ge 0 relation (39) implies that

119896

sum119894=1

10038161003816100381610038161205741198941003816100381610038161003816 ge1205982119896

2119872 (41)

Using Lemma 5 and (40) it follows that

sum119896ge1

(119892119879119896119889119896)2

100381710038171003817100381711988911989610038171003817100381710038172= sum119896ge1

1205742119896

ℎ119896

= +infin (42)

which contradicts to Zoutendijk condition (29) This showsthat (32) holds The proof of the theorem is complete

From the proof of the above theorem we can concludethat any conjugate gradient method with the formula 120573NPRP

+

119896

and some certain step size technique which ensures thatZoutendijk condition (29) holds is globally convergent Inparticular the formula 120573NPRP

+

119896with the weak Wolfe condi-

tions can generate a globally convergent result

4 Numerical Results

All methods above are tested on 56 test problems wherethe former test problems 1ndash48 (from arwhead to woods) inTable 1 are taken from the CUTE library in Bongartz et al[26] and the others are taken fromMore et al [27]

DYHS 120573119896= max 0min 120573HS

119896 120573DY119896 (43)

is generated by Grippo and Lucidi [12]All codes were written in Matlab 75 and run on a HP

with 187GB RAM and Windows XP operating system Theparameters are 120590 = 01 120575 = 001 119906

1= 1 119906

2= 2 and 120582 = 03

Stop the iteration if criterion 119892119896 le 120598 = 10minus6 is satisfied

In Table 1 ldquoNamerdquo denotes the abbreviation of the testproblems ldquo119899rdquo denotes the dimension of the test problemsldquoItrNFNGrdquo denote the number of iteration function eval-uations and gradient evaluations respectively and ldquoTcpurdquodenotes the computing time of CPU for computing a testproblem (units second)

On the other hand to show the performance differenceclearly between the hJHJ hAN hDY and hHuS method weadopt the performance profiles given by Dolan and More[28] to compare the performance according to Itr NF NGand Tcpu respectively Benchmark results are generated byrunning a solver on a set P of problems and recordinginformation of interest such as NF and Tcpu Let S be theset of solvers in comparison Assume that S consists of 119899

119904

0 2 4 6 8 100

02

04

06

08

1

120591

120588(120591)

SWPHS+

SWPPRP+WWPNPRP+

WWPDYHS

Figure 1 Performance profiles on Tpu

0 2 4 6 8 10 120

02

04

06

08

1

120591

120588(120591)

SWPHS+

SWPPRP+WWPNPRP+

WWPDYHS

Figure 2 Performance profile on NF

solvers and P consists of 119899119901problems For each problem

119901 isin P and solver 119904 isin S denote 119905119901119904

by the computing time(or the number of function evaluation etc) required to solveproblem 119901 isin P by solver 119904 isin S and the comparison betweendifferent solvers is based on the performance ratio defined by

119903119901119904=

119905119901119904

min 119905119901119904 119904 isin S

(44)

Assume that a large enough parameter 119903119872ge 119903119901119904

for all119901 119904 is chosen and 119903

119901119904= 119903119872

if and only if solvers 119904 do notsolve problem 119901 Define

120588119904(120591) =

1

119899119901

size 119901 isinP log 119903119901119904le 120591 (45)

where size 119860 means the number of elements in set 119860 then120588119904(120591) is the probability for solver 119904 isin S that a performance

6 Mathematical Problems in Engineering

Table 1 Numerical test report

Name119899 SWPHS+ SWPPRP+ WWPNPRP+ WWPDYHSItrNFNGTcpu ItrNFNGTcpu ItrNFNGTcpu ItrNFNGTcpu

arwhead 1000 10293980078 204548416061462 69260026 23275320073arwhead 2000 164411390184 261682418252753 69760035 23325430135arwhead 10000 319523011554 36611324313917892 8154130219 8138120206chainwoo 1000 11329910058 128374511700632 26305430053 32373440062chnrosnb 20 372130004 372130004 372170004 372130004chnrosnb 50 6149370009 8188580013 589140005 592120006cosine 1000 133681530236 3113380159 30121320048 14001241615343743cragglvy 5000 1120026 1120026 1120026 1120026cragglvy 10000 1120027 1120027 1120026 1120026dixmaana 3000 13484228719153185754 5131410535 1219130128 1614160145dixmaanb 3000 1347230253 570150272 71070066 6960058dixmaanc 3000 1449310288 571180286 81280077 91190086dixmaand 3000 1444122168617815 572210289 1116110109 1213120116dixmaani 3000 FFFF 498330403 FFFF FFFFdixmaanj 3000 FFFF 669230305 1086602167612069 15221420293718554dixmaanl 3000 133551341506 371180272 FFFF FFFFdqdrtic 1000 FFFF 180455951225876841 20514702050205 71484710064dqdrtic 3000 FFFF 1805560402289313054 18213011820334 96648960165dqdrtic 5000 FFFF 1805560782306019535 14210291420390 79530790204dqrtic 6000 163190237 163190238 12720085 12720085dqrtic 15000 13020234 13020235 13020234 13020235dqrtic 20000 13120325 13120323 13120325 13120324edensch 1000 1297398521832135054 1202371041764321938 34181390171 58263770265engval1 3000 4100330033 6081869873766380 65470018 17112330042engval1 10000 4100280092 58017809699417446 758100056 19146450155engval1 20000 4100290184 56517366687133912 758100111 20152280304errinros 50 149402914450321 4131236347080986 FFFF FFFFgenhumps 2000 325662450408 222911190237 11102300071 1297250085genhumps 15000 8178611183 142821121767 22117370615 13187561229genhumps 20000 478230691 5102360928 26188731218 437100275genrose 10000 1209361191277535424 263140056 78972977927213 26323652652333genrose 20000 19716010021284117729 264170119 51347545159346 29726762985283nondia 10000 141100028 141100028 12120015 12120013nondia 15000 141100053 141100044 12220026 12220021nondia 20000 14290063 14290061 12220029 12220029nondquar 3000 263110208 1404041171517728 4728966606186 66295610458098nondquar 15000 26581025 57141950729773 19663030116855 8221496139555029penalty1 100 174393180241 6146490063 8626860072 1821180018penalty1 1000 197505177833 12328992655 1149140438 2465290712power1 10000 12820011 12820012 12820012 12820012power1 15000 153120035 153120036 12920017 12920018power1 20000 13020025 13020025 13020025 13020025quartc 1000 154170038 154170037 12220013 12220013quartc 6000 163190236 163190237 12720087 12720085quartc 10000 12920152 12920152 12920152 12920151srosenbr 5000 394220032 190587020022140 39354410131 78180028tridia 100 FFFF FFFF 55350575530372 34830983480223tridia 1000 FFFF FFFF 19322428319323235 14431758014432335bv 2000 1635816120822 434133975234752704 41140636 7072120707140003lin0 10000 1901686643 1901686659 165258180 165258186lin1 3000 184119126 184119109 16026184 16026197lin1 20000 19318122849 19318122946 168282817 168282707pen1 1000 13120780 13120770 13120768 13120771pen1 5000 138217416 138217415 138217425 138217415vardim 1000 16820716 16820685 16820689 16820687vardim 5000 184219569 184220066 184219628 184219720

Mathematical Problems in Engineering 7

0 2 4 6 8 100

02

04

06

08

1

120591

120588(120591)

SWPHS+

SWPPRP+WWPNPRP+

WWPDYHS

Figure 3 Performance profile on NG

0 2 4 6 8 100

02

04

06

08

1

120591

120588(120591)

SWPHS+

SWPPRP+WWPNPRP+

WWPDYHS

Figure 4 Performance profile on Itr

ratio 119903119901119904

is within a factor 120591 isin 119877119899 The 120588119904is the (cumulative)

distribution function for the performance ratio The value of120588119904(1) is the probability that the solver will win over the rest of

the solversBased on the theory of the performance profile above

four performance figures that is Figures 1ndash4 can be gener-ated according to Table 1 From the four figures we can seethat the NPRP is superior to the other three CGMs on thetesting problems

5 Conclusion

In this paper we carefully studied the combination of thevariations of the formulas 120573FR

119896and 120573PRP

119896 We have found

that the new formula possesses the following features (1)120573NPRP

+

119896is a descent sequence without any line search (2) the

new method possesses the sufficient descent property and

converge globally (3) the strategy will restart the iterationautomatically along the steepest descent direction if a neg-ative value of 120573NPRP

119896occurs (4) the initial numerical results

are promising

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

The authors are very grateful to the anonymous referees fortheir useful suggestions and comments which improved thequality of this paper This work is supported by the NaturalScience Foundation of Shanxi Province (2012JQ9004) NewStar Team of Xirsquoan University of Post and Telecommucica-tions (XY201506) Science Foundation of Liaocheng Univer-sity (318011303) and General Project of the National SocialScience Foundation (15BGL014)

References

[1] G Zoutendijk ldquoNonlinear programming computational meth-odsrdquo in Integer and Non-Linear Programming J Abadie Ed pp37ndash86 North-Holland Publishing Amsterdam The Nerther-lands 1970

[2] M Al-Baali ldquoDescent property and global convergence of thefletcher-reeves method with inexact line searchrdquo IMA Journalof Numerical Analysis vol 5 no 1 pp 121ndash124 1985

[3] Y H Dai and Y Yuan ldquoAn efficient hybrid conjugate gradientmethod for unconstrained optimizationrdquo Annals of OperationsResearch vol 103 no 1ndash4 pp 33ndash47 2001

[4] Z X Wei G Y Li and L Q Qi ldquoGlobal convergence ofthe Polak-Ribiere-Polyak conjugate gradient method with anArmijo-type inexact line search for nonconvex unconstrainedoptimization problemsrdquo Mathematics of Computation vol 77no 264 pp 2173ndash2193 2008

[5] Y H Dai and Y Yuan ldquoA nonlinear conjugate gradient methodwith a strong global convergence propertyrdquo SIAM Journal onOptimization vol 10 no 1 pp 177ndash182 2000

[6] S W Yao Z XWei and H Huang ldquoA note aboutWYLrsquos conju-gate gradientmethod and its applicationsrdquoAppliedMathematicsand Computation vol 191 no 2 pp 381ndash388 2007

[7] X Z Jiang G D Ma and J B Jian ldquoA new global convergentconjugate gradient method with Wolfe line searchrdquo ChineseJournal of Engineering Mathematics vol 28 no 6 pp 779ndash7862011

[8] X Z Jiang L Han and J B Jian ldquoA globally convergentmixed conjugate gradient method with Wolfe line searchrdquoMathematica Numerica Sinica vol 34 no 1 pp 103ndash112 2012

[9] D Touati-Ahmed and C Storey ldquoEfficient hybrid conjugategradient techniquesrdquo Journal of OptimizationTheory and Appli-cations vol 64 no 2 pp 379ndash397 1990

[10] J C Gilbert and J Nocedal ldquoGlobal convergence properties ofconjugate gradient methods for optimizationrdquo SIAM Journal onOptimization vol 2 no 1 pp 21ndash42 1992

[11] Y F Hu and C Storey ldquoGlobal convergence result for conjugategradient methodsrdquo Journal of OptimizationTheory and Applica-tions vol 71 no 2 pp 399ndash405 1991

8 Mathematical Problems in Engineering

[12] L Grippo and S Lucidi ldquoA globally convergent version ofthe polak-ribiere conjugate gradient methodrdquo MathematicalProgramming Series B vol 78 no 3 pp 375ndash391 1997

[13] G H Yu L T Guan and G Y Li ldquoGlobal convergenceof modified Polak-Ribiere-Polyak conjugate gradient methodswith sufficient descent propertyrdquo Journal of Industrial andManagement Optimization vol 4 no 3 pp 565ndash579 2008

[14] N Andrei ldquoNumerical comparison of conjugate gradient algo-rithms for unconstrained optimizationrdquo Studies in Informaticsamp Control vol 16 no 4 pp 333ndash352 2007

[15] Y H Dai Analyses of conjugate gradient methods [PhD thesis]Mathematics and ScientificEngineering Computing ChineseAcademy of Sciences 1997

[16] G Yu L Guan and G Li ldquoGlobal convergence of modifiedPolak-Ribiere-Polyak conjugate gradient methods with suffi-cient descent propertyrdquo Journal of Industrial and ManagementOptimization vol 4 no 3 pp 565ndash579 2008

[17] Y-H Dai and C-X Kou ldquoA nonlinear conjugate gradientalgorithmwith an optimal property and an improved wolfe linesearchrdquo SIAM Journal on Optimization vol 23 no 1 pp 296ndash320 2013

[18] X-Z Jiang and J-B Jian ldquoTwo modified nonlinear conjugategradient methods with disturbance factors for unconstrainedoptimizationrdquoNonlinear Dynamics vol 77 no 1-2 pp 387ndash3972014

[19] J B Jian L Han and X Z Jiang ldquoA hybrid conjugate gradientmethodwith descent property for unconstrained optimizationrdquoApplied Mathematical Modelling vol 39 pp 1281ndash1290 2015

[20] Z Wei G Li and L Qi ldquoNew nonlinear conjugate gradientformulas for large-scale unconstrained optimization problemsrdquoApplied Mathematics and Computation vol 179 no 2 pp 407ndash430 2006

[21] H Huang Z Wei and Y Shengwei ldquoThe proof of the suffi-cient descent condition of the Wei-Yao-Liu conjugate gradientmethod under the strong Wolfe-Powell line searchrdquo AppliedMathematics and Computation vol 189 no 2 pp 1241ndash12452007

[22] G Yu Y Zhao and Z Wei ldquoA descent nonlinear conjugategradient method for large-scale unconstrained optimizationrdquoApplied Mathematics and Computation vol 187 no 2 pp 636ndash643 2007

[23] P Wolfe ldquoConvergence conditions for ascent methodsrdquo SIAMReview vol 11 no 2 pp 226ndash235 1969

[24] P Wolfe ldquoConvergence conditions for ascent methods ii somecorrectionsrdquo SIAM Review vol 13 no 2 pp 185ndash188 1971

[25] Y Dai and Y YuanNonlinear Conjugate Methods Science Pressof Shanghai Shanghai China 2000

[26] I Bongartz A R Conn N Gould and P L Toint ldquoCUTEconstrained and unconstrained testing environmentrdquo ACMTransactions on Mathematical Software vol 21 no 1 pp 123ndash160 1995

[27] J J More B S Garbow and K E Hillstrom ldquoTestingunconstrained optimization softwarerdquo ACM Transactions onMathematical Software vol 7 no 1 pp 17ndash41 1981

[28] E D Dolan and J J More ldquoBenchmarking optimization soft-ware with performance profilesrdquo Mathematical ProgrammingSeries B vol 91 no 2 pp 201ndash213 2002

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of