Research Article An Efficient Hybrid Conjugate Gradient ...(PRP) formula. It reveals a solution for the PRP case which is not globally convergent with the strong Wolfe-Powell (SWP)
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Research ArticleAn Efficient Hybrid Conjugate Gradient Method withthe Strong Wolfe-Powell Line Search
Ahmad Alhawarat1 Mustafa Mamat2 Mohd Rivaie3 and Zabidin Salleh1
1School of Informatics and Applied Mathematics Universiti Malaysia Terengganu 21030 Kuala Terengganu Terengganu Malaysia2Faculty of Informatics and Computing Universiti Sultan Zainal Abidin 21300 Kuala Terengganu Terengganu Malaysia3Department of Computer Science and Mathematics Universiti Teknologi MARA (UITM) Terengganu Campus Kuala Terengganu21080 Kuala Terengganu Terengganu Malaysia
Correspondence should be addressed to Ahmad Alhawarat abadee2010yahoocom
Received 11 March 2015 Revised 6 July 2015 Accepted 7 July 2015
Academic Editor Haipeng Peng
Copyright copy 2015 Ahmad Alhawarat et al This is an open access article distributed under the Creative Commons AttributionLicense which permits unrestricted use distribution and reproduction in any medium provided the original work is properlycited
Conjugate gradient (CG) method is an interesting tool to solve optimization problems in many fields such as design economicsphysics and engineering In this paper we depict a new hybrid of CG method which relates to the famous Polak-Ribiere-Polyak(PRP) formula It reveals a solution for the PRP case which is not globally convergent with the strong Wolfe-Powell (SWP) linesearch The new formula possesses the sufficient descent condition and the global convergent properties In addition we furtherexplained about the cases where PRP method failed with SWP line search Furthermore we provide numerical computations forthe new hybrid CG method which is almost better than other related PRP formulas in both the number of iterations and the CPUtime under some standard test functions
1 Introduction
The nonlinear conjugate gradient (CG) method is a usefultool to find the minimum value for unconstrained optimiza-tion problems Consider the following form
min 119891 (119909) | 119909 isin 119877119899 (1)
where 119891 119877119899 rarr 119877 is continuously differentiable functionand its gradient is denoted by 119892(119909) = nabla119891(119909) The CGmethodis to find a sequence of points 119909
119896 119896 ge 1 starting from
initial point 1199091 isin 119877119899 which is given as the following iterative
In fact (4) is not an effective line search since it needsheavy computations in function and gradient evaluationsTherefore we prefer to use inexpensive line searchThe strongWolfe-Powell (SWP) line search [1 2] which is given asfollows
119891 (119909119896+120572119896119889119896) le 119891 (119909
The CG method has been developed recently based on itssimplicity numerical efficiency and low memory require-ments Thus it is used widely in engineering medical scienceand other fields As an application in engineering we canuse CG method to solve some real life problem similar tothat mentioned in [3] The CG method is limited for thefunctions where their gradient is availableThus the heuristicalgorithm [4] can be used as an alternative method to findthe solution for general functions A heuristic algorithm isto find an approximation solution for the objective functionswith accepted time In addition the heuristic algorithms canbe applied without using computers We refer the reader tosee some applications for this algorithm in [5ndash7]
The most popular formulas for 120573119896are Hestenes-Stiefel
(HS) [8] Fletcher-Reeves (FR) [9] Polak-Ribiere-Polyak(PRP) [10] andWei et al (WYL) [11] respectively as follows
where 119910119896minus1 = 119892119896 minus 119892119896minus1
Hestenes-Stiefel [8] proposed the first formula for solvingthe quadratic functions in 1952 Fletcher and Reeves [9]presented the first formula (9) for nonlinear functions in1964 The convergence properties of FR method with exactline search were obtained by Zoutendijk [12] Al-Baali [13]proved that FR method is globally convergent with the SWPline search when 120590 lt 12 Later on Guanghui et al [14]extended the result to120590 le 12The global convergence of PRPmethod (10) with the exact line search was proved by Elijahand Ribiere in [10] Powell [15] gave out a counterexampleshowing that there exists nonconvex function where PRPmethod does not converge globally even when the exactline search is used Powell suggested using nonnegative PRPmethod to reveal this problem Gilbert and Nocedal [16]proved that nonnegative PRP (PRP+) method that is 120573
119896=
max120573PRP119896 0 is globally convergent under complicated line
searches However there is no guarantee that PRP+ is conver-gent with SWP line search for general nonlinear functions
Touati-Ahmed and Storey [17] suggest the following hybridmethod
120573TS119896=
120573PRP119896 if 0 le 120573PRP
119896le 120573FR119896
120573FR119896 otherwise
(12)
In 2006 Wei et al [11] presented a new positive CG method(11) which is quite similar to original PRP method whichhas been studied in both exact and inexact line search Manymodifications have appeared such as the following [18ndash20]respectively
Recently many CG formulas were constructed in order to getthe efficiency and robustness For more about the latest CGmethods we refer the reader to see [21 22]
One of the important rules in CG methods is the descentcondition that is if one can prove
119892119879119896119889119896lt 0 (14)
then we have a guarantee for119891(119909119896+1) lt 119891(119909119896) If we extended
(14) to the following form
119892119879119896119889119896le minus 119888 1003817100381710038171003817119892119896
10038171003817100381710038172 119896 ge 0 119888 gt 0 (15)
then (15) is called the sufficient descent conditionThis paper is organized as follows in Section 2 we will
present the current problem for PRP and nonnegative PRPmethod with SWP line search Later on we will suggest thenew hybrid CG formula and its simplifications Furthermorewe will establish the global convergence properties withthe SWP line search in Section 3 Numerical results withconclusion will be presented in Sections 4 and 5 respectively
2 Motivation and the Hybrid Formula
The PRP formula is one of the best CG methods in thiscentury However as we mentioned before this methodfails to solve some standard test problems for nonconvexfunctions even the exact line search is used Thus the maincontribution of this paper is to extend using PRP formula inseveral cases with SWP line search under mild condition andrestart the CG algorithm by using NPRP CG formula whenPRP failed to satisfy that condition
The following discussion illustrates the cases in whichPRP method fails and succeeds with SWP line search to
Mathematical Problems in Engineering 3
obtain the convergence propertiesThe PRPmethod could besimplified as follows
In Case B there is no guarantee that this method will satisfythe sufficient descent condition
For the next discussion we will discuss the nonnegativePRP method which is given as follows
120573PRP+119896
= max 120573PRP119896 0 ie
120573PRP+119896
= max Case A1 Case A2 Case B 0 (18)
Therefore we have a problem inCase B To solve this problemGilbert and Nocedal [16] used another line search to satisfythe convergence properties In addition if 120573PRP+
119896= 0 the
CG method returns to the steepest descent method whichis sometimes a weak tool to find the optimum point forfunctions Furthermore we can notice that
120573TS119896=
Case A1
120573FR119896 otherwise
(19)
So from PRP method we can use only Case A1To improve the above ideas we suggest the following
Thus NPRP method is a suitable nonnegative value to useThe following algorithm is an algorithm of CG method
with the new coefficient 120573PRPlowastlowast
119896
Algorithm 1 Consider the followingStep 1 Initialization given 1199091 set 119896 = 1Step 2 Compute 120573
119896based on (20)
Step 3 Compute 119889119896based on (3)
Step 4 Compute 120572119896based on (5) and (6)
Step 5 Update new point based on (2)Step 6 Convergent test and stopping criteria if 119892
119896 le 120576 then
stop otherwise go to Step 2 with 119896 = 119896 + 1
4 Mathematical Problems in Engineering
3 The Global Convergence Properties for120573PRP
lowastlowast
119896with SWP Line Search
The following standard assumptions are necessary for thiswork
Assumption 1 The level set Χ = 119909 isin 119877119899 119891(119909) le 119891(1199091)with 1199091 to the starting point of the iterative method (2) isbounded
Assumption 2 In some open convex neighborhood119873 ofΧ119891is continuous and differentiable and its gradient is Lipschitzcontinuous that is for any 119909 119910 isin 119873 there exists a constant119871 gt 0 such that 119892(119909) minus 119892(119910) le 119871119909 minus 119910
The following lemma is one of the most important lem-maswhich is used to prove the global convergence properties
Lemma 2 (see [12]) Suppose Assumptions 1 and 2 are trueConsider any form of (2) and (3) with 120572
119896computed by WWP
line search direction 119889119896 is descent for all 119896 ge 1 theninfin
100381710038171003817100381711989211989610038171003817100381710038172 le minus 2+
119896minus1sum119895=0(2120590)119895 (30)
Since119896minus1sum119895=0(2120590)119895 le 1 minus (2120590)119896
1 minus 2120590 (31)
we have
minus1 minus (2120590)119896
1 minus 2120590le119892119879119896119889119896
100381710038171003817100381711989211989610038171003817100381710038172 le minus 2+
1 minus (2120590)119896
1 minus 2120590 (32)
If 120590 le 14 we get
1 minus (2120590)119896
1 minus 2120590lt 2 (33)
Let
119888 = 2minus 1 minus (120590)119896
1 minus 120590 (34)
then
119888 minus 2 le119892119879119896119889119896
100381710038171003817100381711989211989610038171003817100381710038172 le minus 119888 (35)
Thus
119892119879119896119889119896le minus 119888 1003817100381710038171003817119892119896
10038171003817100381710038172 (36)
where 119888 isin (0 1) The proof is complete
Gilbert andNocedal [16] presented an important theorem(Theorem 4) to find the global convergence properties fornonnegative PRPmethod if the descent condition is satisfiedFurthermore [16] presented nice property called Propertylowast asfollows
119875119903119900119901119890119903119905119910lowast Consider a CG method of forms (2) and (3) andsuppose 0 lt 120574 le 119892
119896 le 120574 we say that the CG method
possesses Propertylowast if there exists constant 119887 gt 1 and 120582 gt 0such that for all 119896 ge 1 we get |120573
119896| le 119887 and if 119909
119896minus119909119896minus1 le 120582
we obtain
10038161003816100381610038161205731198961003816100381610038161003816 le
12119887 (37)
Theorem 4 Consider any CG method of forms (2) and (3)achieves the following properties
(I) 120573119896ge 0
(II) The sufficient descent condition (15) holds
Mathematical Problems in Engineering 5
(III) The Zoutendijk condition (25) is satisfied by the linesearch
The next lemma shows that if the gradients are boundedaway from zero and Propertylowast holds then a certain fractionof steps cannot be too small The proof is given in [16]However we state it for readability
Lemma 5 Consider a CG algorithm as defined in (2) and (3)with the parameter 120573PRP
lowastlowast
119896 If Assumptions 1 and 2 are satisfied
then 119875119903119900119901119890119903119905119910lowast holds
Proof Let 119887 = 212057421205742 ge 1 and 120582 le 12057422119871120574119887
1003816100381610038161003816100381610038161003816100381610038161003816le100381710038171003817100381711989211989610038171003817100381710038171003817100381710038171003817119892119896 minus 119892119896minus1
Lemma 6 The CG formula presented in Case 2 has thefollowing properties
(1) 120573PRPlowastlowast
119896gt 0 since the condition 119892
1198962 gt |119892119879
119896119892119896minus1| forces
the CG formula in (20) to be nonnegative
(2) 120573PRPlowastlowast
119896satisfies 119875119903119900119901119890119903119905119910lowast based on Lemma 5
(3) 120573PRPlowastlowast
119896satisfies the sufficient descent condition based on
Theorem 3 and 119888 isin (0 1)
By using Theorems 3 and 4 and Lemma 6 we havethe following convergence result The proof is similar toTheorem 43 which is presented in [16]
Theorem 7 Suppose that Assumption 1 holds Consider theCG method of forms (2) and (3) and 120573
119896as in Case 2
where 120572119896is computed by (5) and (6) with 120590 le 14 then
lim119896rarrinfin
inf 119892119896 = 0
t
e0
e1
e2
e3
Ps(t)
00
02
04
06
08
10
FR
FR
NPRPWYLVHS
VHSNPRPWYL
PRPlowastlowast
PRPlowastlowast
PRP+
PRP+
Figure 1 Performance profile based on the number of iterations
4 Numerical Results and Discussions
To evaluate the efficiency of the new method we selectedsome of the test functions in Table 1 from CUTEr [24]Neculai [23] and Adorio and Diliman [25] We performed acomparison with other CG methods including VHS NPRPPRP+ FR and PRPlowastlowast formulas The tolerance 120576 is selectedto 10minus6 for all algorithms to investigate the rapidity of theiterationmethods towards the optimal solutionThe gradientvalue is used as the stopping criteria Here the stoppingcriteria considered 119892
119896 le 10minus6 We considered the method
failed if the number of iterations exceeds 1000 timesIn Table 1 we selected different initial points for every
function Thus this demonstrated that this method can beused in several real life functions from other fields such asengineering and medical science as we mentioned before Inaddition different dimensions from 500 until 10000 are usedWe also choose from different group of functions We usedMatlab 79 subroutine program with CPU processor Intel (R)Core (TM) i3 CPU and 2GB DDR2 RAM under SWP linesearch to find the optimumpointThe efficiency comparisonsresults are shown in Figures 1 and 2 respectively using aperformance profile introduced by Dolan and More [26]
This performance measure was introduced to compare aset of solvers 119878 on a set of problems 119875 Assuming 119899
119904solvers
and 119899119901problems in 119878 and 119875 respectively the measure 119905
119901119904
is defined as the computation time (eg the number ofiterations or the CPU time) required to solve problem 119901 bysolver 119904
To create a baseline for comparison the performance ofsolver 119904 on problem119901 is scaled by the best performance of anysolver in 119878 on the problem using the ratio
119903119901119904=
119905119901119904
min 119905119901119904 119904 isin 119878
(41)
6 Mathematical Problems in Engineering
Table 1 A list of problem functions used with the SWP condition with 120575 = 001 and 120590 = 01
Suppose that a parameter 119903119872ge 119903119901119904
for all 119901 119904 is chosen and119903119901119904= 119903119872
if and only if solver 119904 does not solve problem 119901Because we would like to obtain an overall assessment of theperformance of a solver we defined the measure
Thus 120588119904(119905) is the probability for solver 119904 isin 119878 that the
performance ratio 119903119901119904
is within a factor 119905 isin 119877 of the bestpossible ratio If we define the function 119901
119904as the cumulative
distribution function for the performance ratio then theperformance measure 119901
119904 119877 rarr [0 1] for a solver is
nondecreasing and piecewise continuous from the right Thevalue of 119901
119904(1) is the probability that the solver has the best
performance of all of the solvers In general a solverwith highvalues of 119901(119905) which would appear in the upper right cornerof the figure is preferable
Based on the left side of Figures 1 and 2 the PRPlowastlowastformula is above the other curves Therefore it is the most
efficient method among related PRP methods in terms ofefficiency and robustness In Figure 2 we see that the curveof PRPlowastlowast is still the best but the efficiency is not goodas the number of iterations since we use the complicatedhybrid algorithm leads to high CPU time Thus using highprocessors computers to find the solution will be moreefficient since the number of iterations decreased rapidlyunder PRPlowastlowast method
5 Conclusion
In this paper we proposed hybrid conjugate gradient methodby using nonnegative PRP and NPRP formulas with the SWPline search which extended the cases of using PRP methodunder mild condition The global convergence property isestablished and it is very simple Our numerical results hadshown that the hybrid method is the best when compared toother related PRP CG methods
Mathematical Problems in Engineering 7
t
e0
e1
e2
e3
Ps(t)
00
02
04
06
08
10
FR
FR
NPRP
VHSNPRPWYL
WYLVHS
PRPlowastlowast
PRPlowastlowast
PRP+
PRP+
Figure 2 Performance profile based on the CPU time
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
The authors are grateful to the editor and the anony-mous reviewers for their comments and suggestions whichimproved this paper substantially They would also like tothank The Ministry of Education Malaysia (MOE) for fund-ing this research under The Fundamental Research GrantScheme (Grant no 59256)
References
[1] P Wolfe ldquoConvergence conditions for ascent methodsrdquo SIAMReview vol 11 no 2 pp 226ndash235 1969
[2] PWolfe ldquoConvergence conditions for ascent methods II somecorrectionsrdquo SIAM Review vol 13 no 2 pp 185ndash188 1971
[3] J CaiQ Li L LiH Peng andYYang ldquoA fuzzy adaptive chaoticant swarm optimization for economic dispatchrdquo InternationalJournal of Electrical Power and Energy Systems vol 34 no 1 pp154ndash160 2012
[4] KNatalliaAn Introduction toHeuristic Slgorithms Departmentof Informatics and Telecommunications 2005
[5] L LiH Peng J Kurths Y Yang andH J Schellnhuber ldquoChaos-order transition in foraging behavior of antsrdquo Proceedings of theNational Academy of Sciences of the United States of Americavol 111 no 23 pp 8392ndash8397 2014
[6] M Wan L Li J Xiao C Wang and Y Yang ldquoData clusteringusing bacterial foraging optimizationrdquo Journal of IntelligentInformation Systems vol 38 no 2 pp 321ndash341 2012
[7] M Wan C Wang L Li and Y Yang ldquoChaotic ant swarmapproach for data clusteringrdquo Applied Soft Computing Journalvol 12 no 8 pp 2387ndash2393 2012
[8] M R Hestenes and E Stiefel ldquoMethods of conjugate gradientsfor solving linear systemsrdquo Journal of Research of the NationalBureau of Standards vol 49 no 6 pp 409ndash436 1952
[9] R Fletcher and C M Reeves ldquoFunction minimization byconjugate gradientsrdquo The Computer Journal vol 7 no 2 pp149ndash154 1964
[10] P Elijah and G Ribiere ldquoNote sur la convergence de methodesde directions conjugueesrdquo Revue francaise drsquoinformatique et derecherche operationnelle vol 3 no 1 pp 35ndash43 1969
[11] Z Wei S Yao and L Liu ldquoThe convergence properties of somenew conjugate gradient methodsrdquo Applied Mathematics andComputation vol 183 no 2 pp 1341ndash1350 2006
[12] G Zoutendijk ldquoNonlinear programming computationalmeth-odsrdquo Integer and Nonlinear Programming vol 143 no 1 pp 37ndash86 1970
[13] M Al-Baali ldquoDescent property and global convergence ofthe FletchermdashReeves method with inexact line searchrdquo IMAJournal of Numerical Analysis vol 5 no 1 pp 121ndash124 1985
[14] L Guanghui H Jiye and Y Hongxia ldquoGlobal convergence ofthe fletcher-reeves algorithm with inexact linesearchrdquo AppliedMathematics-A Journal of Chinese Universities vol 10 no 1 pp75ndash82 1995
[15] M J D Powell ldquoNonconvex minimization calculations and theconjugate gradient methodrdquo in Numerical Analysis vol 1066of Lecture Notes in Mathematics pp 122ndash141 Springer BerlinGermany 1984
[16] J C Gilbert and J Nocedal ldquoGlobal convergence properties ofconjugate gradient methods for optimizationrdquo SIAM Journal onOptimization vol 2 no 1 pp 21ndash42 1992
[17] D Touati-Ahmed and C Storey ldquoEfficient hybrid conjugategradient techniquesrdquo Journal of OptimizationTheory and Appli-cations vol 64 no 2 pp 379ndash397 1990
[18] Y Shengwei ZWei andH Huang ldquoA note aboutWYLrsquos conju-gate gradientmethod and its applicationsrdquoAppliedMathematicsand Computation vol 191 no 2 pp 381ndash388 2007
[19] L Zhang ldquoAn improved Wei-Yao-Liu nonlinear conjugategradient method for optimization computationrdquoApplied Math-ematics and Computation vol 215 no 6 pp 2269ndash2274 2009
[20] Z Dai and F Wen ldquoAnother improvedWeindashYaondashLiu nonlinearconjugate gradient method with sufficient descent propertyrdquoAppliedMathematics andComputation vol 218 no 14 pp 7421ndash7430 2012
[21] M Rivaie M Mamat L W June and I Mohd ldquoA new classof nonlinear conjugate gradient coefficients with global conver-gence propertiesrdquo Applied Mathematics and Computation vol218 no 22 pp 11323ndash11332 2012
[22] A AlhawaratMMamatM Rivaie and IMohd ldquoA newmodi-fication of nonlinear conjugate gradient coefficients with globalconvergence propertiesrdquo International Journal of MathematicalComputational Statistical Natural and Physical Engineeringvol 8 no 1 pp 54ndash60 2014
[23] A Neculai ldquoAn unconstrained optimization test functionscollectionrdquo Advanced Modeling and Optimization vol 10 no 1pp 147ndash161 2008
[24] I Bongartz A R Conn N Gould P L Toint and I Bon-gartz Constrained and Unconstrained Testing EnvironmentDepartement de Mathematique 1993
[25] E P Adorio and U Diliman ldquoMvf-multivariate test functionslibrary in c for unconstrained global optimizationrdquo 2005
[26] E D Dolan and J J More ldquoBenchmarking optimization soft-ware with performance profilesrdquo Mathematical Programmingvol 91 no 2 pp 201ndash213 2002
The CG method has been developed recently based on itssimplicity numerical efficiency and low memory require-ments Thus it is used widely in engineering medical scienceand other fields As an application in engineering we canuse CG method to solve some real life problem similar tothat mentioned in [3] The CG method is limited for thefunctions where their gradient is availableThus the heuristicalgorithm [4] can be used as an alternative method to findthe solution for general functions A heuristic algorithm isto find an approximation solution for the objective functionswith accepted time In addition the heuristic algorithms canbe applied without using computers We refer the reader tosee some applications for this algorithm in [5ndash7]
The most popular formulas for 120573119896are Hestenes-Stiefel
(HS) [8] Fletcher-Reeves (FR) [9] Polak-Ribiere-Polyak(PRP) [10] andWei et al (WYL) [11] respectively as follows
where 119910119896minus1 = 119892119896 minus 119892119896minus1
Hestenes-Stiefel [8] proposed the first formula for solvingthe quadratic functions in 1952 Fletcher and Reeves [9]presented the first formula (9) for nonlinear functions in1964 The convergence properties of FR method with exactline search were obtained by Zoutendijk [12] Al-Baali [13]proved that FR method is globally convergent with the SWPline search when 120590 lt 12 Later on Guanghui et al [14]extended the result to120590 le 12The global convergence of PRPmethod (10) with the exact line search was proved by Elijahand Ribiere in [10] Powell [15] gave out a counterexampleshowing that there exists nonconvex function where PRPmethod does not converge globally even when the exactline search is used Powell suggested using nonnegative PRPmethod to reveal this problem Gilbert and Nocedal [16]proved that nonnegative PRP (PRP+) method that is 120573
119896=
max120573PRP119896 0 is globally convergent under complicated line
searches However there is no guarantee that PRP+ is conver-gent with SWP line search for general nonlinear functions
Touati-Ahmed and Storey [17] suggest the following hybridmethod
120573TS119896=
120573PRP119896 if 0 le 120573PRP
119896le 120573FR119896
120573FR119896 otherwise
(12)
In 2006 Wei et al [11] presented a new positive CG method(11) which is quite similar to original PRP method whichhas been studied in both exact and inexact line search Manymodifications have appeared such as the following [18ndash20]respectively
Recently many CG formulas were constructed in order to getthe efficiency and robustness For more about the latest CGmethods we refer the reader to see [21 22]
One of the important rules in CG methods is the descentcondition that is if one can prove
119892119879119896119889119896lt 0 (14)
then we have a guarantee for119891(119909119896+1) lt 119891(119909119896) If we extended
(14) to the following form
119892119879119896119889119896le minus 119888 1003817100381710038171003817119892119896
10038171003817100381710038172 119896 ge 0 119888 gt 0 (15)
then (15) is called the sufficient descent conditionThis paper is organized as follows in Section 2 we will
present the current problem for PRP and nonnegative PRPmethod with SWP line search Later on we will suggest thenew hybrid CG formula and its simplifications Furthermorewe will establish the global convergence properties withthe SWP line search in Section 3 Numerical results withconclusion will be presented in Sections 4 and 5 respectively
2 Motivation and the Hybrid Formula
The PRP formula is one of the best CG methods in thiscentury However as we mentioned before this methodfails to solve some standard test problems for nonconvexfunctions even the exact line search is used Thus the maincontribution of this paper is to extend using PRP formula inseveral cases with SWP line search under mild condition andrestart the CG algorithm by using NPRP CG formula whenPRP failed to satisfy that condition
The following discussion illustrates the cases in whichPRP method fails and succeeds with SWP line search to
Mathematical Problems in Engineering 3
obtain the convergence propertiesThe PRPmethod could besimplified as follows
In Case B there is no guarantee that this method will satisfythe sufficient descent condition
For the next discussion we will discuss the nonnegativePRP method which is given as follows
120573PRP+119896
= max 120573PRP119896 0 ie
120573PRP+119896
= max Case A1 Case A2 Case B 0 (18)
Therefore we have a problem inCase B To solve this problemGilbert and Nocedal [16] used another line search to satisfythe convergence properties In addition if 120573PRP+
119896= 0 the
CG method returns to the steepest descent method whichis sometimes a weak tool to find the optimum point forfunctions Furthermore we can notice that
120573TS119896=
Case A1
120573FR119896 otherwise
(19)
So from PRP method we can use only Case A1To improve the above ideas we suggest the following
Thus NPRP method is a suitable nonnegative value to useThe following algorithm is an algorithm of CG method
with the new coefficient 120573PRPlowastlowast
119896
Algorithm 1 Consider the followingStep 1 Initialization given 1199091 set 119896 = 1Step 2 Compute 120573
119896based on (20)
Step 3 Compute 119889119896based on (3)
Step 4 Compute 120572119896based on (5) and (6)
Step 5 Update new point based on (2)Step 6 Convergent test and stopping criteria if 119892
119896 le 120576 then
stop otherwise go to Step 2 with 119896 = 119896 + 1
4 Mathematical Problems in Engineering
3 The Global Convergence Properties for120573PRP
lowastlowast
119896with SWP Line Search
The following standard assumptions are necessary for thiswork
Assumption 1 The level set Χ = 119909 isin 119877119899 119891(119909) le 119891(1199091)with 1199091 to the starting point of the iterative method (2) isbounded
Assumption 2 In some open convex neighborhood119873 ofΧ119891is continuous and differentiable and its gradient is Lipschitzcontinuous that is for any 119909 119910 isin 119873 there exists a constant119871 gt 0 such that 119892(119909) minus 119892(119910) le 119871119909 minus 119910
The following lemma is one of the most important lem-maswhich is used to prove the global convergence properties
Lemma 2 (see [12]) Suppose Assumptions 1 and 2 are trueConsider any form of (2) and (3) with 120572
119896computed by WWP
line search direction 119889119896 is descent for all 119896 ge 1 theninfin
100381710038171003817100381711989211989610038171003817100381710038172 le minus 2+
119896minus1sum119895=0(2120590)119895 (30)
Since119896minus1sum119895=0(2120590)119895 le 1 minus (2120590)119896
1 minus 2120590 (31)
we have
minus1 minus (2120590)119896
1 minus 2120590le119892119879119896119889119896
100381710038171003817100381711989211989610038171003817100381710038172 le minus 2+
1 minus (2120590)119896
1 minus 2120590 (32)
If 120590 le 14 we get
1 minus (2120590)119896
1 minus 2120590lt 2 (33)
Let
119888 = 2minus 1 minus (120590)119896
1 minus 120590 (34)
then
119888 minus 2 le119892119879119896119889119896
100381710038171003817100381711989211989610038171003817100381710038172 le minus 119888 (35)
Thus
119892119879119896119889119896le minus 119888 1003817100381710038171003817119892119896
10038171003817100381710038172 (36)
where 119888 isin (0 1) The proof is complete
Gilbert andNocedal [16] presented an important theorem(Theorem 4) to find the global convergence properties fornonnegative PRPmethod if the descent condition is satisfiedFurthermore [16] presented nice property called Propertylowast asfollows
119875119903119900119901119890119903119905119910lowast Consider a CG method of forms (2) and (3) andsuppose 0 lt 120574 le 119892
119896 le 120574 we say that the CG method
possesses Propertylowast if there exists constant 119887 gt 1 and 120582 gt 0such that for all 119896 ge 1 we get |120573
119896| le 119887 and if 119909
119896minus119909119896minus1 le 120582
we obtain
10038161003816100381610038161205731198961003816100381610038161003816 le
12119887 (37)
Theorem 4 Consider any CG method of forms (2) and (3)achieves the following properties
(I) 120573119896ge 0
(II) The sufficient descent condition (15) holds
Mathematical Problems in Engineering 5
(III) The Zoutendijk condition (25) is satisfied by the linesearch
The next lemma shows that if the gradients are boundedaway from zero and Propertylowast holds then a certain fractionof steps cannot be too small The proof is given in [16]However we state it for readability
Lemma 5 Consider a CG algorithm as defined in (2) and (3)with the parameter 120573PRP
lowastlowast
119896 If Assumptions 1 and 2 are satisfied
then 119875119903119900119901119890119903119905119910lowast holds
Proof Let 119887 = 212057421205742 ge 1 and 120582 le 12057422119871120574119887
1003816100381610038161003816100381610038161003816100381610038161003816le100381710038171003817100381711989211989610038171003817100381710038171003817100381710038171003817119892119896 minus 119892119896minus1
Lemma 6 The CG formula presented in Case 2 has thefollowing properties
(1) 120573PRPlowastlowast
119896gt 0 since the condition 119892
1198962 gt |119892119879
119896119892119896minus1| forces
the CG formula in (20) to be nonnegative
(2) 120573PRPlowastlowast
119896satisfies 119875119903119900119901119890119903119905119910lowast based on Lemma 5
(3) 120573PRPlowastlowast
119896satisfies the sufficient descent condition based on
Theorem 3 and 119888 isin (0 1)
By using Theorems 3 and 4 and Lemma 6 we havethe following convergence result The proof is similar toTheorem 43 which is presented in [16]
Theorem 7 Suppose that Assumption 1 holds Consider theCG method of forms (2) and (3) and 120573
119896as in Case 2
where 120572119896is computed by (5) and (6) with 120590 le 14 then
lim119896rarrinfin
inf 119892119896 = 0
t
e0
e1
e2
e3
Ps(t)
00
02
04
06
08
10
FR
FR
NPRPWYLVHS
VHSNPRPWYL
PRPlowastlowast
PRPlowastlowast
PRP+
PRP+
Figure 1 Performance profile based on the number of iterations
4 Numerical Results and Discussions
To evaluate the efficiency of the new method we selectedsome of the test functions in Table 1 from CUTEr [24]Neculai [23] and Adorio and Diliman [25] We performed acomparison with other CG methods including VHS NPRPPRP+ FR and PRPlowastlowast formulas The tolerance 120576 is selectedto 10minus6 for all algorithms to investigate the rapidity of theiterationmethods towards the optimal solutionThe gradientvalue is used as the stopping criteria Here the stoppingcriteria considered 119892
119896 le 10minus6 We considered the method
failed if the number of iterations exceeds 1000 timesIn Table 1 we selected different initial points for every
function Thus this demonstrated that this method can beused in several real life functions from other fields such asengineering and medical science as we mentioned before Inaddition different dimensions from 500 until 10000 are usedWe also choose from different group of functions We usedMatlab 79 subroutine program with CPU processor Intel (R)Core (TM) i3 CPU and 2GB DDR2 RAM under SWP linesearch to find the optimumpointThe efficiency comparisonsresults are shown in Figures 1 and 2 respectively using aperformance profile introduced by Dolan and More [26]
This performance measure was introduced to compare aset of solvers 119878 on a set of problems 119875 Assuming 119899
119904solvers
and 119899119901problems in 119878 and 119875 respectively the measure 119905
119901119904
is defined as the computation time (eg the number ofiterations or the CPU time) required to solve problem 119901 bysolver 119904
To create a baseline for comparison the performance ofsolver 119904 on problem119901 is scaled by the best performance of anysolver in 119878 on the problem using the ratio
119903119901119904=
119905119901119904
min 119905119901119904 119904 isin 119878
(41)
6 Mathematical Problems in Engineering
Table 1 A list of problem functions used with the SWP condition with 120575 = 001 and 120590 = 01
Suppose that a parameter 119903119872ge 119903119901119904
for all 119901 119904 is chosen and119903119901119904= 119903119872
if and only if solver 119904 does not solve problem 119901Because we would like to obtain an overall assessment of theperformance of a solver we defined the measure
Thus 120588119904(119905) is the probability for solver 119904 isin 119878 that the
performance ratio 119903119901119904
is within a factor 119905 isin 119877 of the bestpossible ratio If we define the function 119901
119904as the cumulative
distribution function for the performance ratio then theperformance measure 119901
119904 119877 rarr [0 1] for a solver is
nondecreasing and piecewise continuous from the right Thevalue of 119901
119904(1) is the probability that the solver has the best
performance of all of the solvers In general a solverwith highvalues of 119901(119905) which would appear in the upper right cornerof the figure is preferable
Based on the left side of Figures 1 and 2 the PRPlowastlowastformula is above the other curves Therefore it is the most
efficient method among related PRP methods in terms ofefficiency and robustness In Figure 2 we see that the curveof PRPlowastlowast is still the best but the efficiency is not goodas the number of iterations since we use the complicatedhybrid algorithm leads to high CPU time Thus using highprocessors computers to find the solution will be moreefficient since the number of iterations decreased rapidlyunder PRPlowastlowast method
5 Conclusion
In this paper we proposed hybrid conjugate gradient methodby using nonnegative PRP and NPRP formulas with the SWPline search which extended the cases of using PRP methodunder mild condition The global convergence property isestablished and it is very simple Our numerical results hadshown that the hybrid method is the best when compared toother related PRP CG methods
Mathematical Problems in Engineering 7
t
e0
e1
e2
e3
Ps(t)
00
02
04
06
08
10
FR
FR
NPRP
VHSNPRPWYL
WYLVHS
PRPlowastlowast
PRPlowastlowast
PRP+
PRP+
Figure 2 Performance profile based on the CPU time
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
The authors are grateful to the editor and the anony-mous reviewers for their comments and suggestions whichimproved this paper substantially They would also like tothank The Ministry of Education Malaysia (MOE) for fund-ing this research under The Fundamental Research GrantScheme (Grant no 59256)
References
[1] P Wolfe ldquoConvergence conditions for ascent methodsrdquo SIAMReview vol 11 no 2 pp 226ndash235 1969
[2] PWolfe ldquoConvergence conditions for ascent methods II somecorrectionsrdquo SIAM Review vol 13 no 2 pp 185ndash188 1971
[3] J CaiQ Li L LiH Peng andYYang ldquoA fuzzy adaptive chaoticant swarm optimization for economic dispatchrdquo InternationalJournal of Electrical Power and Energy Systems vol 34 no 1 pp154ndash160 2012
[4] KNatalliaAn Introduction toHeuristic Slgorithms Departmentof Informatics and Telecommunications 2005
[5] L LiH Peng J Kurths Y Yang andH J Schellnhuber ldquoChaos-order transition in foraging behavior of antsrdquo Proceedings of theNational Academy of Sciences of the United States of Americavol 111 no 23 pp 8392ndash8397 2014
[6] M Wan L Li J Xiao C Wang and Y Yang ldquoData clusteringusing bacterial foraging optimizationrdquo Journal of IntelligentInformation Systems vol 38 no 2 pp 321ndash341 2012
[7] M Wan C Wang L Li and Y Yang ldquoChaotic ant swarmapproach for data clusteringrdquo Applied Soft Computing Journalvol 12 no 8 pp 2387ndash2393 2012
[8] M R Hestenes and E Stiefel ldquoMethods of conjugate gradientsfor solving linear systemsrdquo Journal of Research of the NationalBureau of Standards vol 49 no 6 pp 409ndash436 1952
[9] R Fletcher and C M Reeves ldquoFunction minimization byconjugate gradientsrdquo The Computer Journal vol 7 no 2 pp149ndash154 1964
[10] P Elijah and G Ribiere ldquoNote sur la convergence de methodesde directions conjugueesrdquo Revue francaise drsquoinformatique et derecherche operationnelle vol 3 no 1 pp 35ndash43 1969
[11] Z Wei S Yao and L Liu ldquoThe convergence properties of somenew conjugate gradient methodsrdquo Applied Mathematics andComputation vol 183 no 2 pp 1341ndash1350 2006
[12] G Zoutendijk ldquoNonlinear programming computationalmeth-odsrdquo Integer and Nonlinear Programming vol 143 no 1 pp 37ndash86 1970
[13] M Al-Baali ldquoDescent property and global convergence ofthe FletchermdashReeves method with inexact line searchrdquo IMAJournal of Numerical Analysis vol 5 no 1 pp 121ndash124 1985
[14] L Guanghui H Jiye and Y Hongxia ldquoGlobal convergence ofthe fletcher-reeves algorithm with inexact linesearchrdquo AppliedMathematics-A Journal of Chinese Universities vol 10 no 1 pp75ndash82 1995
[15] M J D Powell ldquoNonconvex minimization calculations and theconjugate gradient methodrdquo in Numerical Analysis vol 1066of Lecture Notes in Mathematics pp 122ndash141 Springer BerlinGermany 1984
[16] J C Gilbert and J Nocedal ldquoGlobal convergence properties ofconjugate gradient methods for optimizationrdquo SIAM Journal onOptimization vol 2 no 1 pp 21ndash42 1992
[17] D Touati-Ahmed and C Storey ldquoEfficient hybrid conjugategradient techniquesrdquo Journal of OptimizationTheory and Appli-cations vol 64 no 2 pp 379ndash397 1990
[18] Y Shengwei ZWei andH Huang ldquoA note aboutWYLrsquos conju-gate gradientmethod and its applicationsrdquoAppliedMathematicsand Computation vol 191 no 2 pp 381ndash388 2007
[19] L Zhang ldquoAn improved Wei-Yao-Liu nonlinear conjugategradient method for optimization computationrdquoApplied Math-ematics and Computation vol 215 no 6 pp 2269ndash2274 2009
[20] Z Dai and F Wen ldquoAnother improvedWeindashYaondashLiu nonlinearconjugate gradient method with sufficient descent propertyrdquoAppliedMathematics andComputation vol 218 no 14 pp 7421ndash7430 2012
[21] M Rivaie M Mamat L W June and I Mohd ldquoA new classof nonlinear conjugate gradient coefficients with global conver-gence propertiesrdquo Applied Mathematics and Computation vol218 no 22 pp 11323ndash11332 2012
[22] A AlhawaratMMamatM Rivaie and IMohd ldquoA newmodi-fication of nonlinear conjugate gradient coefficients with globalconvergence propertiesrdquo International Journal of MathematicalComputational Statistical Natural and Physical Engineeringvol 8 no 1 pp 54ndash60 2014
[23] A Neculai ldquoAn unconstrained optimization test functionscollectionrdquo Advanced Modeling and Optimization vol 10 no 1pp 147ndash161 2008
[24] I Bongartz A R Conn N Gould P L Toint and I Bon-gartz Constrained and Unconstrained Testing EnvironmentDepartement de Mathematique 1993
[25] E P Adorio and U Diliman ldquoMvf-multivariate test functionslibrary in c for unconstrained global optimizationrdquo 2005
[26] E D Dolan and J J More ldquoBenchmarking optimization soft-ware with performance profilesrdquo Mathematical Programmingvol 91 no 2 pp 201ndash213 2002
In Case B there is no guarantee that this method will satisfythe sufficient descent condition
For the next discussion we will discuss the nonnegativePRP method which is given as follows
120573PRP+119896
= max 120573PRP119896 0 ie
120573PRP+119896
= max Case A1 Case A2 Case B 0 (18)
Therefore we have a problem inCase B To solve this problemGilbert and Nocedal [16] used another line search to satisfythe convergence properties In addition if 120573PRP+
119896= 0 the
CG method returns to the steepest descent method whichis sometimes a weak tool to find the optimum point forfunctions Furthermore we can notice that
120573TS119896=
Case A1
120573FR119896 otherwise
(19)
So from PRP method we can use only Case A1To improve the above ideas we suggest the following
Thus NPRP method is a suitable nonnegative value to useThe following algorithm is an algorithm of CG method
with the new coefficient 120573PRPlowastlowast
119896
Algorithm 1 Consider the followingStep 1 Initialization given 1199091 set 119896 = 1Step 2 Compute 120573
119896based on (20)
Step 3 Compute 119889119896based on (3)
Step 4 Compute 120572119896based on (5) and (6)
Step 5 Update new point based on (2)Step 6 Convergent test and stopping criteria if 119892
119896 le 120576 then
stop otherwise go to Step 2 with 119896 = 119896 + 1
4 Mathematical Problems in Engineering
3 The Global Convergence Properties for120573PRP
lowastlowast
119896with SWP Line Search
The following standard assumptions are necessary for thiswork
Assumption 1 The level set Χ = 119909 isin 119877119899 119891(119909) le 119891(1199091)with 1199091 to the starting point of the iterative method (2) isbounded
Assumption 2 In some open convex neighborhood119873 ofΧ119891is continuous and differentiable and its gradient is Lipschitzcontinuous that is for any 119909 119910 isin 119873 there exists a constant119871 gt 0 such that 119892(119909) minus 119892(119910) le 119871119909 minus 119910
The following lemma is one of the most important lem-maswhich is used to prove the global convergence properties
Lemma 2 (see [12]) Suppose Assumptions 1 and 2 are trueConsider any form of (2) and (3) with 120572
119896computed by WWP
line search direction 119889119896 is descent for all 119896 ge 1 theninfin
100381710038171003817100381711989211989610038171003817100381710038172 le minus 2+
119896minus1sum119895=0(2120590)119895 (30)
Since119896minus1sum119895=0(2120590)119895 le 1 minus (2120590)119896
1 minus 2120590 (31)
we have
minus1 minus (2120590)119896
1 minus 2120590le119892119879119896119889119896
100381710038171003817100381711989211989610038171003817100381710038172 le minus 2+
1 minus (2120590)119896
1 minus 2120590 (32)
If 120590 le 14 we get
1 minus (2120590)119896
1 minus 2120590lt 2 (33)
Let
119888 = 2minus 1 minus (120590)119896
1 minus 120590 (34)
then
119888 minus 2 le119892119879119896119889119896
100381710038171003817100381711989211989610038171003817100381710038172 le minus 119888 (35)
Thus
119892119879119896119889119896le minus 119888 1003817100381710038171003817119892119896
10038171003817100381710038172 (36)
where 119888 isin (0 1) The proof is complete
Gilbert andNocedal [16] presented an important theorem(Theorem 4) to find the global convergence properties fornonnegative PRPmethod if the descent condition is satisfiedFurthermore [16] presented nice property called Propertylowast asfollows
119875119903119900119901119890119903119905119910lowast Consider a CG method of forms (2) and (3) andsuppose 0 lt 120574 le 119892
119896 le 120574 we say that the CG method
possesses Propertylowast if there exists constant 119887 gt 1 and 120582 gt 0such that for all 119896 ge 1 we get |120573
119896| le 119887 and if 119909
119896minus119909119896minus1 le 120582
we obtain
10038161003816100381610038161205731198961003816100381610038161003816 le
12119887 (37)
Theorem 4 Consider any CG method of forms (2) and (3)achieves the following properties
(I) 120573119896ge 0
(II) The sufficient descent condition (15) holds
Mathematical Problems in Engineering 5
(III) The Zoutendijk condition (25) is satisfied by the linesearch
The next lemma shows that if the gradients are boundedaway from zero and Propertylowast holds then a certain fractionof steps cannot be too small The proof is given in [16]However we state it for readability
Lemma 5 Consider a CG algorithm as defined in (2) and (3)with the parameter 120573PRP
lowastlowast
119896 If Assumptions 1 and 2 are satisfied
then 119875119903119900119901119890119903119905119910lowast holds
Proof Let 119887 = 212057421205742 ge 1 and 120582 le 12057422119871120574119887
1003816100381610038161003816100381610038161003816100381610038161003816le100381710038171003817100381711989211989610038171003817100381710038171003817100381710038171003817119892119896 minus 119892119896minus1
Lemma 6 The CG formula presented in Case 2 has thefollowing properties
(1) 120573PRPlowastlowast
119896gt 0 since the condition 119892
1198962 gt |119892119879
119896119892119896minus1| forces
the CG formula in (20) to be nonnegative
(2) 120573PRPlowastlowast
119896satisfies 119875119903119900119901119890119903119905119910lowast based on Lemma 5
(3) 120573PRPlowastlowast
119896satisfies the sufficient descent condition based on
Theorem 3 and 119888 isin (0 1)
By using Theorems 3 and 4 and Lemma 6 we havethe following convergence result The proof is similar toTheorem 43 which is presented in [16]
Theorem 7 Suppose that Assumption 1 holds Consider theCG method of forms (2) and (3) and 120573
119896as in Case 2
where 120572119896is computed by (5) and (6) with 120590 le 14 then
lim119896rarrinfin
inf 119892119896 = 0
t
e0
e1
e2
e3
Ps(t)
00
02
04
06
08
10
FR
FR
NPRPWYLVHS
VHSNPRPWYL
PRPlowastlowast
PRPlowastlowast
PRP+
PRP+
Figure 1 Performance profile based on the number of iterations
4 Numerical Results and Discussions
To evaluate the efficiency of the new method we selectedsome of the test functions in Table 1 from CUTEr [24]Neculai [23] and Adorio and Diliman [25] We performed acomparison with other CG methods including VHS NPRPPRP+ FR and PRPlowastlowast formulas The tolerance 120576 is selectedto 10minus6 for all algorithms to investigate the rapidity of theiterationmethods towards the optimal solutionThe gradientvalue is used as the stopping criteria Here the stoppingcriteria considered 119892
119896 le 10minus6 We considered the method
failed if the number of iterations exceeds 1000 timesIn Table 1 we selected different initial points for every
function Thus this demonstrated that this method can beused in several real life functions from other fields such asengineering and medical science as we mentioned before Inaddition different dimensions from 500 until 10000 are usedWe also choose from different group of functions We usedMatlab 79 subroutine program with CPU processor Intel (R)Core (TM) i3 CPU and 2GB DDR2 RAM under SWP linesearch to find the optimumpointThe efficiency comparisonsresults are shown in Figures 1 and 2 respectively using aperformance profile introduced by Dolan and More [26]
This performance measure was introduced to compare aset of solvers 119878 on a set of problems 119875 Assuming 119899
119904solvers
and 119899119901problems in 119878 and 119875 respectively the measure 119905
119901119904
is defined as the computation time (eg the number ofiterations or the CPU time) required to solve problem 119901 bysolver 119904
To create a baseline for comparison the performance ofsolver 119904 on problem119901 is scaled by the best performance of anysolver in 119878 on the problem using the ratio
119903119901119904=
119905119901119904
min 119905119901119904 119904 isin 119878
(41)
6 Mathematical Problems in Engineering
Table 1 A list of problem functions used with the SWP condition with 120575 = 001 and 120590 = 01
Suppose that a parameter 119903119872ge 119903119901119904
for all 119901 119904 is chosen and119903119901119904= 119903119872
if and only if solver 119904 does not solve problem 119901Because we would like to obtain an overall assessment of theperformance of a solver we defined the measure
Thus 120588119904(119905) is the probability for solver 119904 isin 119878 that the
performance ratio 119903119901119904
is within a factor 119905 isin 119877 of the bestpossible ratio If we define the function 119901
119904as the cumulative
distribution function for the performance ratio then theperformance measure 119901
119904 119877 rarr [0 1] for a solver is
nondecreasing and piecewise continuous from the right Thevalue of 119901
119904(1) is the probability that the solver has the best
performance of all of the solvers In general a solverwith highvalues of 119901(119905) which would appear in the upper right cornerof the figure is preferable
Based on the left side of Figures 1 and 2 the PRPlowastlowastformula is above the other curves Therefore it is the most
efficient method among related PRP methods in terms ofefficiency and robustness In Figure 2 we see that the curveof PRPlowastlowast is still the best but the efficiency is not goodas the number of iterations since we use the complicatedhybrid algorithm leads to high CPU time Thus using highprocessors computers to find the solution will be moreefficient since the number of iterations decreased rapidlyunder PRPlowastlowast method
5 Conclusion
In this paper we proposed hybrid conjugate gradient methodby using nonnegative PRP and NPRP formulas with the SWPline search which extended the cases of using PRP methodunder mild condition The global convergence property isestablished and it is very simple Our numerical results hadshown that the hybrid method is the best when compared toother related PRP CG methods
Mathematical Problems in Engineering 7
t
e0
e1
e2
e3
Ps(t)
00
02
04
06
08
10
FR
FR
NPRP
VHSNPRPWYL
WYLVHS
PRPlowastlowast
PRPlowastlowast
PRP+
PRP+
Figure 2 Performance profile based on the CPU time
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
The authors are grateful to the editor and the anony-mous reviewers for their comments and suggestions whichimproved this paper substantially They would also like tothank The Ministry of Education Malaysia (MOE) for fund-ing this research under The Fundamental Research GrantScheme (Grant no 59256)
References
[1] P Wolfe ldquoConvergence conditions for ascent methodsrdquo SIAMReview vol 11 no 2 pp 226ndash235 1969
[2] PWolfe ldquoConvergence conditions for ascent methods II somecorrectionsrdquo SIAM Review vol 13 no 2 pp 185ndash188 1971
[3] J CaiQ Li L LiH Peng andYYang ldquoA fuzzy adaptive chaoticant swarm optimization for economic dispatchrdquo InternationalJournal of Electrical Power and Energy Systems vol 34 no 1 pp154ndash160 2012
[4] KNatalliaAn Introduction toHeuristic Slgorithms Departmentof Informatics and Telecommunications 2005
[5] L LiH Peng J Kurths Y Yang andH J Schellnhuber ldquoChaos-order transition in foraging behavior of antsrdquo Proceedings of theNational Academy of Sciences of the United States of Americavol 111 no 23 pp 8392ndash8397 2014
[6] M Wan L Li J Xiao C Wang and Y Yang ldquoData clusteringusing bacterial foraging optimizationrdquo Journal of IntelligentInformation Systems vol 38 no 2 pp 321ndash341 2012
[7] M Wan C Wang L Li and Y Yang ldquoChaotic ant swarmapproach for data clusteringrdquo Applied Soft Computing Journalvol 12 no 8 pp 2387ndash2393 2012
[8] M R Hestenes and E Stiefel ldquoMethods of conjugate gradientsfor solving linear systemsrdquo Journal of Research of the NationalBureau of Standards vol 49 no 6 pp 409ndash436 1952
[9] R Fletcher and C M Reeves ldquoFunction minimization byconjugate gradientsrdquo The Computer Journal vol 7 no 2 pp149ndash154 1964
[10] P Elijah and G Ribiere ldquoNote sur la convergence de methodesde directions conjugueesrdquo Revue francaise drsquoinformatique et derecherche operationnelle vol 3 no 1 pp 35ndash43 1969
[11] Z Wei S Yao and L Liu ldquoThe convergence properties of somenew conjugate gradient methodsrdquo Applied Mathematics andComputation vol 183 no 2 pp 1341ndash1350 2006
[12] G Zoutendijk ldquoNonlinear programming computationalmeth-odsrdquo Integer and Nonlinear Programming vol 143 no 1 pp 37ndash86 1970
[13] M Al-Baali ldquoDescent property and global convergence ofthe FletchermdashReeves method with inexact line searchrdquo IMAJournal of Numerical Analysis vol 5 no 1 pp 121ndash124 1985
[14] L Guanghui H Jiye and Y Hongxia ldquoGlobal convergence ofthe fletcher-reeves algorithm with inexact linesearchrdquo AppliedMathematics-A Journal of Chinese Universities vol 10 no 1 pp75ndash82 1995
[15] M J D Powell ldquoNonconvex minimization calculations and theconjugate gradient methodrdquo in Numerical Analysis vol 1066of Lecture Notes in Mathematics pp 122ndash141 Springer BerlinGermany 1984
[16] J C Gilbert and J Nocedal ldquoGlobal convergence properties ofconjugate gradient methods for optimizationrdquo SIAM Journal onOptimization vol 2 no 1 pp 21ndash42 1992
[17] D Touati-Ahmed and C Storey ldquoEfficient hybrid conjugategradient techniquesrdquo Journal of OptimizationTheory and Appli-cations vol 64 no 2 pp 379ndash397 1990
[18] Y Shengwei ZWei andH Huang ldquoA note aboutWYLrsquos conju-gate gradientmethod and its applicationsrdquoAppliedMathematicsand Computation vol 191 no 2 pp 381ndash388 2007
[19] L Zhang ldquoAn improved Wei-Yao-Liu nonlinear conjugategradient method for optimization computationrdquoApplied Math-ematics and Computation vol 215 no 6 pp 2269ndash2274 2009
[20] Z Dai and F Wen ldquoAnother improvedWeindashYaondashLiu nonlinearconjugate gradient method with sufficient descent propertyrdquoAppliedMathematics andComputation vol 218 no 14 pp 7421ndash7430 2012
[21] M Rivaie M Mamat L W June and I Mohd ldquoA new classof nonlinear conjugate gradient coefficients with global conver-gence propertiesrdquo Applied Mathematics and Computation vol218 no 22 pp 11323ndash11332 2012
[22] A AlhawaratMMamatM Rivaie and IMohd ldquoA newmodi-fication of nonlinear conjugate gradient coefficients with globalconvergence propertiesrdquo International Journal of MathematicalComputational Statistical Natural and Physical Engineeringvol 8 no 1 pp 54ndash60 2014
[23] A Neculai ldquoAn unconstrained optimization test functionscollectionrdquo Advanced Modeling and Optimization vol 10 no 1pp 147ndash161 2008
[24] I Bongartz A R Conn N Gould P L Toint and I Bon-gartz Constrained and Unconstrained Testing EnvironmentDepartement de Mathematique 1993
[25] E P Adorio and U Diliman ldquoMvf-multivariate test functionslibrary in c for unconstrained global optimizationrdquo 2005
[26] E D Dolan and J J More ldquoBenchmarking optimization soft-ware with performance profilesrdquo Mathematical Programmingvol 91 no 2 pp 201ndash213 2002
The following standard assumptions are necessary for thiswork
Assumption 1 The level set Χ = 119909 isin 119877119899 119891(119909) le 119891(1199091)with 1199091 to the starting point of the iterative method (2) isbounded
Assumption 2 In some open convex neighborhood119873 ofΧ119891is continuous and differentiable and its gradient is Lipschitzcontinuous that is for any 119909 119910 isin 119873 there exists a constant119871 gt 0 such that 119892(119909) minus 119892(119910) le 119871119909 minus 119910
The following lemma is one of the most important lem-maswhich is used to prove the global convergence properties
Lemma 2 (see [12]) Suppose Assumptions 1 and 2 are trueConsider any form of (2) and (3) with 120572
119896computed by WWP
line search direction 119889119896 is descent for all 119896 ge 1 theninfin
100381710038171003817100381711989211989610038171003817100381710038172 le minus 2+
119896minus1sum119895=0(2120590)119895 (30)
Since119896minus1sum119895=0(2120590)119895 le 1 minus (2120590)119896
1 minus 2120590 (31)
we have
minus1 minus (2120590)119896
1 minus 2120590le119892119879119896119889119896
100381710038171003817100381711989211989610038171003817100381710038172 le minus 2+
1 minus (2120590)119896
1 minus 2120590 (32)
If 120590 le 14 we get
1 minus (2120590)119896
1 minus 2120590lt 2 (33)
Let
119888 = 2minus 1 minus (120590)119896
1 minus 120590 (34)
then
119888 minus 2 le119892119879119896119889119896
100381710038171003817100381711989211989610038171003817100381710038172 le minus 119888 (35)
Thus
119892119879119896119889119896le minus 119888 1003817100381710038171003817119892119896
10038171003817100381710038172 (36)
where 119888 isin (0 1) The proof is complete
Gilbert andNocedal [16] presented an important theorem(Theorem 4) to find the global convergence properties fornonnegative PRPmethod if the descent condition is satisfiedFurthermore [16] presented nice property called Propertylowast asfollows
119875119903119900119901119890119903119905119910lowast Consider a CG method of forms (2) and (3) andsuppose 0 lt 120574 le 119892
119896 le 120574 we say that the CG method
possesses Propertylowast if there exists constant 119887 gt 1 and 120582 gt 0such that for all 119896 ge 1 we get |120573
119896| le 119887 and if 119909
119896minus119909119896minus1 le 120582
we obtain
10038161003816100381610038161205731198961003816100381610038161003816 le
12119887 (37)
Theorem 4 Consider any CG method of forms (2) and (3)achieves the following properties
(I) 120573119896ge 0
(II) The sufficient descent condition (15) holds
Mathematical Problems in Engineering 5
(III) The Zoutendijk condition (25) is satisfied by the linesearch
The next lemma shows that if the gradients are boundedaway from zero and Propertylowast holds then a certain fractionof steps cannot be too small The proof is given in [16]However we state it for readability
Lemma 5 Consider a CG algorithm as defined in (2) and (3)with the parameter 120573PRP
lowastlowast
119896 If Assumptions 1 and 2 are satisfied
then 119875119903119900119901119890119903119905119910lowast holds
Proof Let 119887 = 212057421205742 ge 1 and 120582 le 12057422119871120574119887
1003816100381610038161003816100381610038161003816100381610038161003816le100381710038171003817100381711989211989610038171003817100381710038171003817100381710038171003817119892119896 minus 119892119896minus1
Lemma 6 The CG formula presented in Case 2 has thefollowing properties
(1) 120573PRPlowastlowast
119896gt 0 since the condition 119892
1198962 gt |119892119879
119896119892119896minus1| forces
the CG formula in (20) to be nonnegative
(2) 120573PRPlowastlowast
119896satisfies 119875119903119900119901119890119903119905119910lowast based on Lemma 5
(3) 120573PRPlowastlowast
119896satisfies the sufficient descent condition based on
Theorem 3 and 119888 isin (0 1)
By using Theorems 3 and 4 and Lemma 6 we havethe following convergence result The proof is similar toTheorem 43 which is presented in [16]
Theorem 7 Suppose that Assumption 1 holds Consider theCG method of forms (2) and (3) and 120573
119896as in Case 2
where 120572119896is computed by (5) and (6) with 120590 le 14 then
lim119896rarrinfin
inf 119892119896 = 0
t
e0
e1
e2
e3
Ps(t)
00
02
04
06
08
10
FR
FR
NPRPWYLVHS
VHSNPRPWYL
PRPlowastlowast
PRPlowastlowast
PRP+
PRP+
Figure 1 Performance profile based on the number of iterations
4 Numerical Results and Discussions
To evaluate the efficiency of the new method we selectedsome of the test functions in Table 1 from CUTEr [24]Neculai [23] and Adorio and Diliman [25] We performed acomparison with other CG methods including VHS NPRPPRP+ FR and PRPlowastlowast formulas The tolerance 120576 is selectedto 10minus6 for all algorithms to investigate the rapidity of theiterationmethods towards the optimal solutionThe gradientvalue is used as the stopping criteria Here the stoppingcriteria considered 119892
119896 le 10minus6 We considered the method
failed if the number of iterations exceeds 1000 timesIn Table 1 we selected different initial points for every
function Thus this demonstrated that this method can beused in several real life functions from other fields such asengineering and medical science as we mentioned before Inaddition different dimensions from 500 until 10000 are usedWe also choose from different group of functions We usedMatlab 79 subroutine program with CPU processor Intel (R)Core (TM) i3 CPU and 2GB DDR2 RAM under SWP linesearch to find the optimumpointThe efficiency comparisonsresults are shown in Figures 1 and 2 respectively using aperformance profile introduced by Dolan and More [26]
This performance measure was introduced to compare aset of solvers 119878 on a set of problems 119875 Assuming 119899
119904solvers
and 119899119901problems in 119878 and 119875 respectively the measure 119905
119901119904
is defined as the computation time (eg the number ofiterations or the CPU time) required to solve problem 119901 bysolver 119904
To create a baseline for comparison the performance ofsolver 119904 on problem119901 is scaled by the best performance of anysolver in 119878 on the problem using the ratio
119903119901119904=
119905119901119904
min 119905119901119904 119904 isin 119878
(41)
6 Mathematical Problems in Engineering
Table 1 A list of problem functions used with the SWP condition with 120575 = 001 and 120590 = 01
Suppose that a parameter 119903119872ge 119903119901119904
for all 119901 119904 is chosen and119903119901119904= 119903119872
if and only if solver 119904 does not solve problem 119901Because we would like to obtain an overall assessment of theperformance of a solver we defined the measure
Thus 120588119904(119905) is the probability for solver 119904 isin 119878 that the
performance ratio 119903119901119904
is within a factor 119905 isin 119877 of the bestpossible ratio If we define the function 119901
119904as the cumulative
distribution function for the performance ratio then theperformance measure 119901
119904 119877 rarr [0 1] for a solver is
nondecreasing and piecewise continuous from the right Thevalue of 119901
119904(1) is the probability that the solver has the best
performance of all of the solvers In general a solverwith highvalues of 119901(119905) which would appear in the upper right cornerof the figure is preferable
Based on the left side of Figures 1 and 2 the PRPlowastlowastformula is above the other curves Therefore it is the most
efficient method among related PRP methods in terms ofefficiency and robustness In Figure 2 we see that the curveof PRPlowastlowast is still the best but the efficiency is not goodas the number of iterations since we use the complicatedhybrid algorithm leads to high CPU time Thus using highprocessors computers to find the solution will be moreefficient since the number of iterations decreased rapidlyunder PRPlowastlowast method
5 Conclusion
In this paper we proposed hybrid conjugate gradient methodby using nonnegative PRP and NPRP formulas with the SWPline search which extended the cases of using PRP methodunder mild condition The global convergence property isestablished and it is very simple Our numerical results hadshown that the hybrid method is the best when compared toother related PRP CG methods
Mathematical Problems in Engineering 7
t
e0
e1
e2
e3
Ps(t)
00
02
04
06
08
10
FR
FR
NPRP
VHSNPRPWYL
WYLVHS
PRPlowastlowast
PRPlowastlowast
PRP+
PRP+
Figure 2 Performance profile based on the CPU time
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
The authors are grateful to the editor and the anony-mous reviewers for their comments and suggestions whichimproved this paper substantially They would also like tothank The Ministry of Education Malaysia (MOE) for fund-ing this research under The Fundamental Research GrantScheme (Grant no 59256)
References
[1] P Wolfe ldquoConvergence conditions for ascent methodsrdquo SIAMReview vol 11 no 2 pp 226ndash235 1969
[2] PWolfe ldquoConvergence conditions for ascent methods II somecorrectionsrdquo SIAM Review vol 13 no 2 pp 185ndash188 1971
[3] J CaiQ Li L LiH Peng andYYang ldquoA fuzzy adaptive chaoticant swarm optimization for economic dispatchrdquo InternationalJournal of Electrical Power and Energy Systems vol 34 no 1 pp154ndash160 2012
[4] KNatalliaAn Introduction toHeuristic Slgorithms Departmentof Informatics and Telecommunications 2005
[5] L LiH Peng J Kurths Y Yang andH J Schellnhuber ldquoChaos-order transition in foraging behavior of antsrdquo Proceedings of theNational Academy of Sciences of the United States of Americavol 111 no 23 pp 8392ndash8397 2014
[6] M Wan L Li J Xiao C Wang and Y Yang ldquoData clusteringusing bacterial foraging optimizationrdquo Journal of IntelligentInformation Systems vol 38 no 2 pp 321ndash341 2012
[7] M Wan C Wang L Li and Y Yang ldquoChaotic ant swarmapproach for data clusteringrdquo Applied Soft Computing Journalvol 12 no 8 pp 2387ndash2393 2012
[8] M R Hestenes and E Stiefel ldquoMethods of conjugate gradientsfor solving linear systemsrdquo Journal of Research of the NationalBureau of Standards vol 49 no 6 pp 409ndash436 1952
[9] R Fletcher and C M Reeves ldquoFunction minimization byconjugate gradientsrdquo The Computer Journal vol 7 no 2 pp149ndash154 1964
[10] P Elijah and G Ribiere ldquoNote sur la convergence de methodesde directions conjugueesrdquo Revue francaise drsquoinformatique et derecherche operationnelle vol 3 no 1 pp 35ndash43 1969
[11] Z Wei S Yao and L Liu ldquoThe convergence properties of somenew conjugate gradient methodsrdquo Applied Mathematics andComputation vol 183 no 2 pp 1341ndash1350 2006
[12] G Zoutendijk ldquoNonlinear programming computationalmeth-odsrdquo Integer and Nonlinear Programming vol 143 no 1 pp 37ndash86 1970
[13] M Al-Baali ldquoDescent property and global convergence ofthe FletchermdashReeves method with inexact line searchrdquo IMAJournal of Numerical Analysis vol 5 no 1 pp 121ndash124 1985
[14] L Guanghui H Jiye and Y Hongxia ldquoGlobal convergence ofthe fletcher-reeves algorithm with inexact linesearchrdquo AppliedMathematics-A Journal of Chinese Universities vol 10 no 1 pp75ndash82 1995
[15] M J D Powell ldquoNonconvex minimization calculations and theconjugate gradient methodrdquo in Numerical Analysis vol 1066of Lecture Notes in Mathematics pp 122ndash141 Springer BerlinGermany 1984
[16] J C Gilbert and J Nocedal ldquoGlobal convergence properties ofconjugate gradient methods for optimizationrdquo SIAM Journal onOptimization vol 2 no 1 pp 21ndash42 1992
[17] D Touati-Ahmed and C Storey ldquoEfficient hybrid conjugategradient techniquesrdquo Journal of OptimizationTheory and Appli-cations vol 64 no 2 pp 379ndash397 1990
[18] Y Shengwei ZWei andH Huang ldquoA note aboutWYLrsquos conju-gate gradientmethod and its applicationsrdquoAppliedMathematicsand Computation vol 191 no 2 pp 381ndash388 2007
[19] L Zhang ldquoAn improved Wei-Yao-Liu nonlinear conjugategradient method for optimization computationrdquoApplied Math-ematics and Computation vol 215 no 6 pp 2269ndash2274 2009
[20] Z Dai and F Wen ldquoAnother improvedWeindashYaondashLiu nonlinearconjugate gradient method with sufficient descent propertyrdquoAppliedMathematics andComputation vol 218 no 14 pp 7421ndash7430 2012
[21] M Rivaie M Mamat L W June and I Mohd ldquoA new classof nonlinear conjugate gradient coefficients with global conver-gence propertiesrdquo Applied Mathematics and Computation vol218 no 22 pp 11323ndash11332 2012
[22] A AlhawaratMMamatM Rivaie and IMohd ldquoA newmodi-fication of nonlinear conjugate gradient coefficients with globalconvergence propertiesrdquo International Journal of MathematicalComputational Statistical Natural and Physical Engineeringvol 8 no 1 pp 54ndash60 2014
[23] A Neculai ldquoAn unconstrained optimization test functionscollectionrdquo Advanced Modeling and Optimization vol 10 no 1pp 147ndash161 2008
[24] I Bongartz A R Conn N Gould P L Toint and I Bon-gartz Constrained and Unconstrained Testing EnvironmentDepartement de Mathematique 1993
[25] E P Adorio and U Diliman ldquoMvf-multivariate test functionslibrary in c for unconstrained global optimizationrdquo 2005
[26] E D Dolan and J J More ldquoBenchmarking optimization soft-ware with performance profilesrdquo Mathematical Programmingvol 91 no 2 pp 201ndash213 2002
The next lemma shows that if the gradients are boundedaway from zero and Propertylowast holds then a certain fractionof steps cannot be too small The proof is given in [16]However we state it for readability
Lemma 5 Consider a CG algorithm as defined in (2) and (3)with the parameter 120573PRP
lowastlowast
119896 If Assumptions 1 and 2 are satisfied
then 119875119903119900119901119890119903119905119910lowast holds
Proof Let 119887 = 212057421205742 ge 1 and 120582 le 12057422119871120574119887
1003816100381610038161003816100381610038161003816100381610038161003816le100381710038171003817100381711989211989610038171003817100381710038171003817100381710038171003817119892119896 minus 119892119896minus1
Lemma 6 The CG formula presented in Case 2 has thefollowing properties
(1) 120573PRPlowastlowast
119896gt 0 since the condition 119892
1198962 gt |119892119879
119896119892119896minus1| forces
the CG formula in (20) to be nonnegative
(2) 120573PRPlowastlowast
119896satisfies 119875119903119900119901119890119903119905119910lowast based on Lemma 5
(3) 120573PRPlowastlowast
119896satisfies the sufficient descent condition based on
Theorem 3 and 119888 isin (0 1)
By using Theorems 3 and 4 and Lemma 6 we havethe following convergence result The proof is similar toTheorem 43 which is presented in [16]
Theorem 7 Suppose that Assumption 1 holds Consider theCG method of forms (2) and (3) and 120573
119896as in Case 2
where 120572119896is computed by (5) and (6) with 120590 le 14 then
lim119896rarrinfin
inf 119892119896 = 0
t
e0
e1
e2
e3
Ps(t)
00
02
04
06
08
10
FR
FR
NPRPWYLVHS
VHSNPRPWYL
PRPlowastlowast
PRPlowastlowast
PRP+
PRP+
Figure 1 Performance profile based on the number of iterations
4 Numerical Results and Discussions
To evaluate the efficiency of the new method we selectedsome of the test functions in Table 1 from CUTEr [24]Neculai [23] and Adorio and Diliman [25] We performed acomparison with other CG methods including VHS NPRPPRP+ FR and PRPlowastlowast formulas The tolerance 120576 is selectedto 10minus6 for all algorithms to investigate the rapidity of theiterationmethods towards the optimal solutionThe gradientvalue is used as the stopping criteria Here the stoppingcriteria considered 119892
119896 le 10minus6 We considered the method
failed if the number of iterations exceeds 1000 timesIn Table 1 we selected different initial points for every
function Thus this demonstrated that this method can beused in several real life functions from other fields such asengineering and medical science as we mentioned before Inaddition different dimensions from 500 until 10000 are usedWe also choose from different group of functions We usedMatlab 79 subroutine program with CPU processor Intel (R)Core (TM) i3 CPU and 2GB DDR2 RAM under SWP linesearch to find the optimumpointThe efficiency comparisonsresults are shown in Figures 1 and 2 respectively using aperformance profile introduced by Dolan and More [26]
This performance measure was introduced to compare aset of solvers 119878 on a set of problems 119875 Assuming 119899
119904solvers
and 119899119901problems in 119878 and 119875 respectively the measure 119905
119901119904
is defined as the computation time (eg the number ofiterations or the CPU time) required to solve problem 119901 bysolver 119904
To create a baseline for comparison the performance ofsolver 119904 on problem119901 is scaled by the best performance of anysolver in 119878 on the problem using the ratio
119903119901119904=
119905119901119904
min 119905119901119904 119904 isin 119878
(41)
6 Mathematical Problems in Engineering
Table 1 A list of problem functions used with the SWP condition with 120575 = 001 and 120590 = 01
Suppose that a parameter 119903119872ge 119903119901119904
for all 119901 119904 is chosen and119903119901119904= 119903119872
if and only if solver 119904 does not solve problem 119901Because we would like to obtain an overall assessment of theperformance of a solver we defined the measure
Thus 120588119904(119905) is the probability for solver 119904 isin 119878 that the
performance ratio 119903119901119904
is within a factor 119905 isin 119877 of the bestpossible ratio If we define the function 119901
119904as the cumulative
distribution function for the performance ratio then theperformance measure 119901
119904 119877 rarr [0 1] for a solver is
nondecreasing and piecewise continuous from the right Thevalue of 119901
119904(1) is the probability that the solver has the best
performance of all of the solvers In general a solverwith highvalues of 119901(119905) which would appear in the upper right cornerof the figure is preferable
Based on the left side of Figures 1 and 2 the PRPlowastlowastformula is above the other curves Therefore it is the most
efficient method among related PRP methods in terms ofefficiency and robustness In Figure 2 we see that the curveof PRPlowastlowast is still the best but the efficiency is not goodas the number of iterations since we use the complicatedhybrid algorithm leads to high CPU time Thus using highprocessors computers to find the solution will be moreefficient since the number of iterations decreased rapidlyunder PRPlowastlowast method
5 Conclusion
In this paper we proposed hybrid conjugate gradient methodby using nonnegative PRP and NPRP formulas with the SWPline search which extended the cases of using PRP methodunder mild condition The global convergence property isestablished and it is very simple Our numerical results hadshown that the hybrid method is the best when compared toother related PRP CG methods
Mathematical Problems in Engineering 7
t
e0
e1
e2
e3
Ps(t)
00
02
04
06
08
10
FR
FR
NPRP
VHSNPRPWYL
WYLVHS
PRPlowastlowast
PRPlowastlowast
PRP+
PRP+
Figure 2 Performance profile based on the CPU time
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
The authors are grateful to the editor and the anony-mous reviewers for their comments and suggestions whichimproved this paper substantially They would also like tothank The Ministry of Education Malaysia (MOE) for fund-ing this research under The Fundamental Research GrantScheme (Grant no 59256)
References
[1] P Wolfe ldquoConvergence conditions for ascent methodsrdquo SIAMReview vol 11 no 2 pp 226ndash235 1969
[2] PWolfe ldquoConvergence conditions for ascent methods II somecorrectionsrdquo SIAM Review vol 13 no 2 pp 185ndash188 1971
[3] J CaiQ Li L LiH Peng andYYang ldquoA fuzzy adaptive chaoticant swarm optimization for economic dispatchrdquo InternationalJournal of Electrical Power and Energy Systems vol 34 no 1 pp154ndash160 2012
[4] KNatalliaAn Introduction toHeuristic Slgorithms Departmentof Informatics and Telecommunications 2005
[5] L LiH Peng J Kurths Y Yang andH J Schellnhuber ldquoChaos-order transition in foraging behavior of antsrdquo Proceedings of theNational Academy of Sciences of the United States of Americavol 111 no 23 pp 8392ndash8397 2014
[6] M Wan L Li J Xiao C Wang and Y Yang ldquoData clusteringusing bacterial foraging optimizationrdquo Journal of IntelligentInformation Systems vol 38 no 2 pp 321ndash341 2012
[7] M Wan C Wang L Li and Y Yang ldquoChaotic ant swarmapproach for data clusteringrdquo Applied Soft Computing Journalvol 12 no 8 pp 2387ndash2393 2012
[8] M R Hestenes and E Stiefel ldquoMethods of conjugate gradientsfor solving linear systemsrdquo Journal of Research of the NationalBureau of Standards vol 49 no 6 pp 409ndash436 1952
[9] R Fletcher and C M Reeves ldquoFunction minimization byconjugate gradientsrdquo The Computer Journal vol 7 no 2 pp149ndash154 1964
[10] P Elijah and G Ribiere ldquoNote sur la convergence de methodesde directions conjugueesrdquo Revue francaise drsquoinformatique et derecherche operationnelle vol 3 no 1 pp 35ndash43 1969
[11] Z Wei S Yao and L Liu ldquoThe convergence properties of somenew conjugate gradient methodsrdquo Applied Mathematics andComputation vol 183 no 2 pp 1341ndash1350 2006
[12] G Zoutendijk ldquoNonlinear programming computationalmeth-odsrdquo Integer and Nonlinear Programming vol 143 no 1 pp 37ndash86 1970
[13] M Al-Baali ldquoDescent property and global convergence ofthe FletchermdashReeves method with inexact line searchrdquo IMAJournal of Numerical Analysis vol 5 no 1 pp 121ndash124 1985
[14] L Guanghui H Jiye and Y Hongxia ldquoGlobal convergence ofthe fletcher-reeves algorithm with inexact linesearchrdquo AppliedMathematics-A Journal of Chinese Universities vol 10 no 1 pp75ndash82 1995
[15] M J D Powell ldquoNonconvex minimization calculations and theconjugate gradient methodrdquo in Numerical Analysis vol 1066of Lecture Notes in Mathematics pp 122ndash141 Springer BerlinGermany 1984
[16] J C Gilbert and J Nocedal ldquoGlobal convergence properties ofconjugate gradient methods for optimizationrdquo SIAM Journal onOptimization vol 2 no 1 pp 21ndash42 1992
[17] D Touati-Ahmed and C Storey ldquoEfficient hybrid conjugategradient techniquesrdquo Journal of OptimizationTheory and Appli-cations vol 64 no 2 pp 379ndash397 1990
[18] Y Shengwei ZWei andH Huang ldquoA note aboutWYLrsquos conju-gate gradientmethod and its applicationsrdquoAppliedMathematicsand Computation vol 191 no 2 pp 381ndash388 2007
[19] L Zhang ldquoAn improved Wei-Yao-Liu nonlinear conjugategradient method for optimization computationrdquoApplied Math-ematics and Computation vol 215 no 6 pp 2269ndash2274 2009
[20] Z Dai and F Wen ldquoAnother improvedWeindashYaondashLiu nonlinearconjugate gradient method with sufficient descent propertyrdquoAppliedMathematics andComputation vol 218 no 14 pp 7421ndash7430 2012
[21] M Rivaie M Mamat L W June and I Mohd ldquoA new classof nonlinear conjugate gradient coefficients with global conver-gence propertiesrdquo Applied Mathematics and Computation vol218 no 22 pp 11323ndash11332 2012
[22] A AlhawaratMMamatM Rivaie and IMohd ldquoA newmodi-fication of nonlinear conjugate gradient coefficients with globalconvergence propertiesrdquo International Journal of MathematicalComputational Statistical Natural and Physical Engineeringvol 8 no 1 pp 54ndash60 2014
[23] A Neculai ldquoAn unconstrained optimization test functionscollectionrdquo Advanced Modeling and Optimization vol 10 no 1pp 147ndash161 2008
[24] I Bongartz A R Conn N Gould P L Toint and I Bon-gartz Constrained and Unconstrained Testing EnvironmentDepartement de Mathematique 1993
[25] E P Adorio and U Diliman ldquoMvf-multivariate test functionslibrary in c for unconstrained global optimizationrdquo 2005
[26] E D Dolan and J J More ldquoBenchmarking optimization soft-ware with performance profilesrdquo Mathematical Programmingvol 91 no 2 pp 201ndash213 2002
Suppose that a parameter 119903119872ge 119903119901119904
for all 119901 119904 is chosen and119903119901119904= 119903119872
if and only if solver 119904 does not solve problem 119901Because we would like to obtain an overall assessment of theperformance of a solver we defined the measure
Thus 120588119904(119905) is the probability for solver 119904 isin 119878 that the
performance ratio 119903119901119904
is within a factor 119905 isin 119877 of the bestpossible ratio If we define the function 119901
119904as the cumulative
distribution function for the performance ratio then theperformance measure 119901
119904 119877 rarr [0 1] for a solver is
nondecreasing and piecewise continuous from the right Thevalue of 119901
119904(1) is the probability that the solver has the best
performance of all of the solvers In general a solverwith highvalues of 119901(119905) which would appear in the upper right cornerof the figure is preferable
Based on the left side of Figures 1 and 2 the PRPlowastlowastformula is above the other curves Therefore it is the most
efficient method among related PRP methods in terms ofefficiency and robustness In Figure 2 we see that the curveof PRPlowastlowast is still the best but the efficiency is not goodas the number of iterations since we use the complicatedhybrid algorithm leads to high CPU time Thus using highprocessors computers to find the solution will be moreefficient since the number of iterations decreased rapidlyunder PRPlowastlowast method
5 Conclusion
In this paper we proposed hybrid conjugate gradient methodby using nonnegative PRP and NPRP formulas with the SWPline search which extended the cases of using PRP methodunder mild condition The global convergence property isestablished and it is very simple Our numerical results hadshown that the hybrid method is the best when compared toother related PRP CG methods
Mathematical Problems in Engineering 7
t
e0
e1
e2
e3
Ps(t)
00
02
04
06
08
10
FR
FR
NPRP
VHSNPRPWYL
WYLVHS
PRPlowastlowast
PRPlowastlowast
PRP+
PRP+
Figure 2 Performance profile based on the CPU time
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
The authors are grateful to the editor and the anony-mous reviewers for their comments and suggestions whichimproved this paper substantially They would also like tothank The Ministry of Education Malaysia (MOE) for fund-ing this research under The Fundamental Research GrantScheme (Grant no 59256)
References
[1] P Wolfe ldquoConvergence conditions for ascent methodsrdquo SIAMReview vol 11 no 2 pp 226ndash235 1969
[2] PWolfe ldquoConvergence conditions for ascent methods II somecorrectionsrdquo SIAM Review vol 13 no 2 pp 185ndash188 1971
[3] J CaiQ Li L LiH Peng andYYang ldquoA fuzzy adaptive chaoticant swarm optimization for economic dispatchrdquo InternationalJournal of Electrical Power and Energy Systems vol 34 no 1 pp154ndash160 2012
[4] KNatalliaAn Introduction toHeuristic Slgorithms Departmentof Informatics and Telecommunications 2005
[5] L LiH Peng J Kurths Y Yang andH J Schellnhuber ldquoChaos-order transition in foraging behavior of antsrdquo Proceedings of theNational Academy of Sciences of the United States of Americavol 111 no 23 pp 8392ndash8397 2014
[6] M Wan L Li J Xiao C Wang and Y Yang ldquoData clusteringusing bacterial foraging optimizationrdquo Journal of IntelligentInformation Systems vol 38 no 2 pp 321ndash341 2012
[7] M Wan C Wang L Li and Y Yang ldquoChaotic ant swarmapproach for data clusteringrdquo Applied Soft Computing Journalvol 12 no 8 pp 2387ndash2393 2012
[8] M R Hestenes and E Stiefel ldquoMethods of conjugate gradientsfor solving linear systemsrdquo Journal of Research of the NationalBureau of Standards vol 49 no 6 pp 409ndash436 1952
[9] R Fletcher and C M Reeves ldquoFunction minimization byconjugate gradientsrdquo The Computer Journal vol 7 no 2 pp149ndash154 1964
[10] P Elijah and G Ribiere ldquoNote sur la convergence de methodesde directions conjugueesrdquo Revue francaise drsquoinformatique et derecherche operationnelle vol 3 no 1 pp 35ndash43 1969
[11] Z Wei S Yao and L Liu ldquoThe convergence properties of somenew conjugate gradient methodsrdquo Applied Mathematics andComputation vol 183 no 2 pp 1341ndash1350 2006
[12] G Zoutendijk ldquoNonlinear programming computationalmeth-odsrdquo Integer and Nonlinear Programming vol 143 no 1 pp 37ndash86 1970
[13] M Al-Baali ldquoDescent property and global convergence ofthe FletchermdashReeves method with inexact line searchrdquo IMAJournal of Numerical Analysis vol 5 no 1 pp 121ndash124 1985
[14] L Guanghui H Jiye and Y Hongxia ldquoGlobal convergence ofthe fletcher-reeves algorithm with inexact linesearchrdquo AppliedMathematics-A Journal of Chinese Universities vol 10 no 1 pp75ndash82 1995
[15] M J D Powell ldquoNonconvex minimization calculations and theconjugate gradient methodrdquo in Numerical Analysis vol 1066of Lecture Notes in Mathematics pp 122ndash141 Springer BerlinGermany 1984
[16] J C Gilbert and J Nocedal ldquoGlobal convergence properties ofconjugate gradient methods for optimizationrdquo SIAM Journal onOptimization vol 2 no 1 pp 21ndash42 1992
[17] D Touati-Ahmed and C Storey ldquoEfficient hybrid conjugategradient techniquesrdquo Journal of OptimizationTheory and Appli-cations vol 64 no 2 pp 379ndash397 1990
[18] Y Shengwei ZWei andH Huang ldquoA note aboutWYLrsquos conju-gate gradientmethod and its applicationsrdquoAppliedMathematicsand Computation vol 191 no 2 pp 381ndash388 2007
[19] L Zhang ldquoAn improved Wei-Yao-Liu nonlinear conjugategradient method for optimization computationrdquoApplied Math-ematics and Computation vol 215 no 6 pp 2269ndash2274 2009
[20] Z Dai and F Wen ldquoAnother improvedWeindashYaondashLiu nonlinearconjugate gradient method with sufficient descent propertyrdquoAppliedMathematics andComputation vol 218 no 14 pp 7421ndash7430 2012
[21] M Rivaie M Mamat L W June and I Mohd ldquoA new classof nonlinear conjugate gradient coefficients with global conver-gence propertiesrdquo Applied Mathematics and Computation vol218 no 22 pp 11323ndash11332 2012
[22] A AlhawaratMMamatM Rivaie and IMohd ldquoA newmodi-fication of nonlinear conjugate gradient coefficients with globalconvergence propertiesrdquo International Journal of MathematicalComputational Statistical Natural and Physical Engineeringvol 8 no 1 pp 54ndash60 2014
[23] A Neculai ldquoAn unconstrained optimization test functionscollectionrdquo Advanced Modeling and Optimization vol 10 no 1pp 147ndash161 2008
[24] I Bongartz A R Conn N Gould P L Toint and I Bon-gartz Constrained and Unconstrained Testing EnvironmentDepartement de Mathematique 1993
[25] E P Adorio and U Diliman ldquoMvf-multivariate test functionslibrary in c for unconstrained global optimizationrdquo 2005
[26] E D Dolan and J J More ldquoBenchmarking optimization soft-ware with performance profilesrdquo Mathematical Programmingvol 91 no 2 pp 201ndash213 2002
Figure 2 Performance profile based on the CPU time
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
The authors are grateful to the editor and the anony-mous reviewers for their comments and suggestions whichimproved this paper substantially They would also like tothank The Ministry of Education Malaysia (MOE) for fund-ing this research under The Fundamental Research GrantScheme (Grant no 59256)
References
[1] P Wolfe ldquoConvergence conditions for ascent methodsrdquo SIAMReview vol 11 no 2 pp 226ndash235 1969
[2] PWolfe ldquoConvergence conditions for ascent methods II somecorrectionsrdquo SIAM Review vol 13 no 2 pp 185ndash188 1971
[3] J CaiQ Li L LiH Peng andYYang ldquoA fuzzy adaptive chaoticant swarm optimization for economic dispatchrdquo InternationalJournal of Electrical Power and Energy Systems vol 34 no 1 pp154ndash160 2012
[4] KNatalliaAn Introduction toHeuristic Slgorithms Departmentof Informatics and Telecommunications 2005
[5] L LiH Peng J Kurths Y Yang andH J Schellnhuber ldquoChaos-order transition in foraging behavior of antsrdquo Proceedings of theNational Academy of Sciences of the United States of Americavol 111 no 23 pp 8392ndash8397 2014
[6] M Wan L Li J Xiao C Wang and Y Yang ldquoData clusteringusing bacterial foraging optimizationrdquo Journal of IntelligentInformation Systems vol 38 no 2 pp 321ndash341 2012
[7] M Wan C Wang L Li and Y Yang ldquoChaotic ant swarmapproach for data clusteringrdquo Applied Soft Computing Journalvol 12 no 8 pp 2387ndash2393 2012
[8] M R Hestenes and E Stiefel ldquoMethods of conjugate gradientsfor solving linear systemsrdquo Journal of Research of the NationalBureau of Standards vol 49 no 6 pp 409ndash436 1952
[9] R Fletcher and C M Reeves ldquoFunction minimization byconjugate gradientsrdquo The Computer Journal vol 7 no 2 pp149ndash154 1964
[10] P Elijah and G Ribiere ldquoNote sur la convergence de methodesde directions conjugueesrdquo Revue francaise drsquoinformatique et derecherche operationnelle vol 3 no 1 pp 35ndash43 1969
[11] Z Wei S Yao and L Liu ldquoThe convergence properties of somenew conjugate gradient methodsrdquo Applied Mathematics andComputation vol 183 no 2 pp 1341ndash1350 2006
[12] G Zoutendijk ldquoNonlinear programming computationalmeth-odsrdquo Integer and Nonlinear Programming vol 143 no 1 pp 37ndash86 1970
[13] M Al-Baali ldquoDescent property and global convergence ofthe FletchermdashReeves method with inexact line searchrdquo IMAJournal of Numerical Analysis vol 5 no 1 pp 121ndash124 1985
[14] L Guanghui H Jiye and Y Hongxia ldquoGlobal convergence ofthe fletcher-reeves algorithm with inexact linesearchrdquo AppliedMathematics-A Journal of Chinese Universities vol 10 no 1 pp75ndash82 1995
[15] M J D Powell ldquoNonconvex minimization calculations and theconjugate gradient methodrdquo in Numerical Analysis vol 1066of Lecture Notes in Mathematics pp 122ndash141 Springer BerlinGermany 1984
[16] J C Gilbert and J Nocedal ldquoGlobal convergence properties ofconjugate gradient methods for optimizationrdquo SIAM Journal onOptimization vol 2 no 1 pp 21ndash42 1992
[17] D Touati-Ahmed and C Storey ldquoEfficient hybrid conjugategradient techniquesrdquo Journal of OptimizationTheory and Appli-cations vol 64 no 2 pp 379ndash397 1990
[18] Y Shengwei ZWei andH Huang ldquoA note aboutWYLrsquos conju-gate gradientmethod and its applicationsrdquoAppliedMathematicsand Computation vol 191 no 2 pp 381ndash388 2007
[19] L Zhang ldquoAn improved Wei-Yao-Liu nonlinear conjugategradient method for optimization computationrdquoApplied Math-ematics and Computation vol 215 no 6 pp 2269ndash2274 2009
[20] Z Dai and F Wen ldquoAnother improvedWeindashYaondashLiu nonlinearconjugate gradient method with sufficient descent propertyrdquoAppliedMathematics andComputation vol 218 no 14 pp 7421ndash7430 2012
[21] M Rivaie M Mamat L W June and I Mohd ldquoA new classof nonlinear conjugate gradient coefficients with global conver-gence propertiesrdquo Applied Mathematics and Computation vol218 no 22 pp 11323ndash11332 2012
[22] A AlhawaratMMamatM Rivaie and IMohd ldquoA newmodi-fication of nonlinear conjugate gradient coefficients with globalconvergence propertiesrdquo International Journal of MathematicalComputational Statistical Natural and Physical Engineeringvol 8 no 1 pp 54ndash60 2014
[23] A Neculai ldquoAn unconstrained optimization test functionscollectionrdquo Advanced Modeling and Optimization vol 10 no 1pp 147ndash161 2008
[24] I Bongartz A R Conn N Gould P L Toint and I Bon-gartz Constrained and Unconstrained Testing EnvironmentDepartement de Mathematique 1993
[25] E P Adorio and U Diliman ldquoMvf-multivariate test functionslibrary in c for unconstrained global optimizationrdquo 2005
[26] E D Dolan and J J More ldquoBenchmarking optimization soft-ware with performance profilesrdquo Mathematical Programmingvol 91 no 2 pp 201ndash213 2002