This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
RESEARCH Open Access
Modified nonlinear conjugate gradient methodwith sufficient descent condition forunconstrained optimizationJinkui Liu* and Shaoheng Wang
* Correspondence:[email protected] of Mathematics andStatistics, Chongqing Three GorgesUniversity, Chongqing, Wanzhou,People’s Republic of China
Abstract
In this paper, an efficient modified nonlinear conjugate gradient method for solvingunconstrained optimization problems is proposed. An attractive property of themodified method is that the generated direction in each step is always descendingwithout any line search. The global convergence result of the modified method isestablished under the general Wolfe line search condition. Numerical results showthat the modified method is efficient and stationary by comparing with the well-known Polak-Ribiére-Polyak method, CG-DESCENT method and DSP-CG methodusing the unconstrained optimization problems from More and Garbow (ACM TransMath Softw 7, 17-41, 1981), so it can be widely used in scientific computation.Mathematics Subject Classification (2010) 90C26 · 65H10
1 IntroductionThe conjugate gradient method comprises a class of unconstrained optimization algo-
rithms which is characterized by low memory requirements and strong local or global
convergence properties. The purpose of this paper is to study the global convergence
properties and practical computational performance of a modified nonlinear conjugate
gradient method for unconstrained optimization without restarts, and with appropriate
conditions.
In this paper, we consider the unconstrained optimization problem:
min {f (x)|x ∈ Rn}, (1:1)
where f : Rn ® R is a real-valued, continuously differentiable function.
When applied to the nonlinear problem (1.1), a nonlinear conjugate gradient method
generates a sequence {xk}, k ≥ 1, starting from an initial guess x1 Î Rn, using the recur-
rence
xk+1 = xk + αkdk, (1:2)
where the positive step size ak is obtained by some line search, and the search direc-
tion dk is generated by the rule:
dk ={−gk, for k = 1,
−gk + βkdk−1, for k ≥ 2.(1:3)
Liu and Wang Journal of Inequalities and Applications 2011, 2011:57http://www.journalofinequalitiesandapplications.com/content/2011/1/57
Liu and Wang Journal of Inequalities and Applications 2011, 2011:57http://www.journalofinequalitiesandapplications.com/content/2011/1/57
Page 9 of 12
where l is some integer. According to the results on automatic differentiation [20,21],
the value of l can be set to 5, i.e.
Ntota1 = NF + 5 ∗ NG. (4:2)
That is to say, one gradient evaluation is equivalent to five function evaluations if
automatic differentiation is used.
By making used of (4.2), we compare the VLS method with DSP-CG method, PRP
method and CG-DESCENT method as follows: for the ith problem, compute the total
numbers of function evaluations and gradient evaluations required by the VLS method,
DSP-CG method, PRP method and CG-DESCENT method by formula (4.2), and
denote them by Ntotal,i (VLS), Ntotal,i (DSP-CG), Ntotal,i (PRP) and Ntotal,i (CG-DES-
CENT), respectively. Then we calculate the ratio
γi(DSP - CG) =Ntotal,i(DSP - CG)
Ntotal,i(VLS),
γi(PRP) =Ntotal,i(PRP)Ntotal,i(VLS)
,
γi(CG - DESCENT) =Ntotal,i(CG - DESCENT)
Ntotal,i(VLS).
If the i0th problem is not run by the method, we use a constant l = max{gi(method)|i Î S1} instead of γi0 (the method), where S1 denotes the set of the test pro-
blems which can be run by the method. The geometric mean of these ratios for VLS
method over all the test problems is defined by
γ (DSP - CG) =
(∏i∈S
γi(DSP - CG)
) 1|S|
,
γ (PRP) =
(∏i∈S
γi(PRP)
) 1|S|
,
γ (CG - DESCENT) =
(∏i∈S
γi(CG - DESCENT)
) 1|S|
,
where S denotes the set of the test problems, and |S| denotes the number of ele-
ments in S. One advantage of the above rule is that, the comparison is relative and
hence does not be dominated by a few problems for which the method requires a
great deal of function evaluations and gradient functions.
Table 2 The list of the tested problems
N Problem N Problem N Problem N Problem N Problem
1 ROSE 7 BRAD 13 WOOD 19 JNSAM 25 ROSEX
2 FROTH 8 GAUSS 14 KOWOSB 20 VAEDIM 26 SINGX
3 BADSCP 9 MEYER 15 BD 21 WATSON 27 BV
4 BADSCB 10 GULF 16 OSB1 22 PEN2 28 IE
5 BEALE 11 BOX 17 BIGGS 23 PEN1 29 TRID
6 HELIX 12 SING 18 OSB2 24 TRIG
Liu and Wang Journal of Inequalities and Applications 2011, 2011:57http://www.journalofinequalitiesandapplications.com/content/2011/1/57
Page 10 of 12
According to the above rule, it is clear that g (VLS) = 1. The values of g (DSP-CG), g(PRP) and g (CG-DESCENT) are listed in Table 3.
Secondly, we adopt the performance profiles by Dolan and Moré [22] to compare the
VLS method to the DSP-CG method, PRP method and CG-DESCENT method in the
CPU time performance (see Figure 1) In Figure 1,
X = τ , Y = P{log2(rp,s) ≤ τ : 1 ≤ s ≤ ns}.
That is, for each method, we plot the fraction P of problems for which the method is
within a factor τ of the best time. The left side of the figure gives the percentage of the
test problems for which a method is fastest; the right side gives the percentage of the
test problems that were successfully solved by each of the methods. The top curve is
the method that solved the most problems in a time that was within a factor τ of the
best time. Since the top curve in Figure 1 corresponds to VLS method, this method is
clearly fastest for this set for 78 test problems. In particular, the VLS method is fastest
for about 60% of the test problems, and it ultimately solves 100% of the test problems.
From Table 3 and Figure 1, it is clear that the VLS method performs better in the
average performance and the CPU time performance, which implies that the proposed
modified method is computationally efficient.
Table 3 Relative efficiency of the VLS, DSP-CG, PRP and CG-DESCENT methods
Figure 1 Performance profiles of the conjugate gradient methods with respect to CPU time.
Liu and Wang Journal of Inequalities and Applications 2011, 2011:57http://www.journalofinequalitiesandapplications.com/content/2011/1/57
Page 11 of 12
AcknowledgementsThe authors wish to express their heart felt thanks to the referees and Professor K. Teo for their detailed and helpfulsuggestions for revising the manuscript. At the same time, we are grateful for the suggestions of Lijuan Zhang. Thiswork was supported by The Nature Science Foundation of Chongqing Education Committee (KJ091104, KJ101108)and Chongqing Three Gorges University (09ZZ-060).
Authors’ contributionsJinkui Liu carried out the new method studies, designed all the steps of proof in this research and drafted themanuscript. Shaoheng Wang participated in writing the all codes of the algorithm and suggested many good ideasthat made this paper possible. All authors read and approved the final manuscript.
Competing interestsThe authors declare that they have no competing interests.
Received: 3 March 2011 Accepted: 17 September 2011 Published: 17 September 2011
References1. Liu, Y, Storey, C: Efficient generalized conjugate gradient algorithms. Part 1: theory. J Optim Theory Appl. 69, 129–137 (1992)2. Polak, E, Ribire, G: Note sur la convergence de directions conjugees. Rev Francaise Informat Recherche Operatinelle 3e
Annee. 16, 35–43 (1969)3. Polak, BT: The conjugate gradient method in extreme problems. USSR Comput Math Math Phys. 9, 94–112 (1969).
doi:10.1016/0041-5553(69)90035-44. Gaohang, Yu, Yanlin, Zhao, Zengxin, Wei: A descent nonlinear conjugate gradient method for large-scale unconstrained
optimization. Appl Math Comput. 187, 636–643 (2007). doi:10.1016/j.amc.2006.08.0875. Jinkui, Liu, Xianglin, Du, Kairong, Wang: Convergence of descent methods with variable parameters. Acta Math Appl Sin
33, 222–230 (2010). (in Chinese)6. Hager, WW, Zhang, H: A survey of nonlinear conjugate gradient methods. Pac J Optim. 2, 35–58 (2006)7. Powell, MJD: Nonconvex minimization calculations and the conjugate gradient method. In Numerical Analysis (Dundee,
1983). Lecture Notes in Mathematics, vol. 1066, pp. 122–141.Springer, Berlin (1984)8. Andrei, N: Scaled conjugate gradient algorithms for unconstrained optimization. Comput Optim Appl. 38, 401–416
(2007). doi:10.1007/s10589-007-9055-79. Andrei, N: Another nonlinear conjugate gradient algorithm for unconstrained optimization. Optim Methods Softw. 24,
89–104 (2009). doi:10.1080/1055678080239332610. Birgin, EG, Martínez, JM: A spectral conjugate gradient method for unconstrained optimization. Appl Math Optim. 43,
117–128 (2001). doi:10.1007/s00245-001-0003-011. Dai, Y-H, Liao, L-Z: New conjugacy conditions and related nonlinear conjugate gradient methods. Appl Math Optim. 43,
87–101 (2001). doi:10.1007/s00245001001912. Hager, WW, Zhang, H: A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM J
Optim. 16, 170–192 (2005). doi:10.1137/03060188013. Li, G, Tang, C, Wei, Z: New conjugacy condition and related new conjugate gradient methods for unconstrained
optimization. J Comput Appl Math. 202, 523–539 (2007). doi:10.1016/j.cam.2006.03.00514. Wei, Z, Li, G, Qi, L: New quasi-Newton methods for unconstrained optimization problems. Appl Math Comput. 175,
1156–1188 (2006). doi:10.1016/j.amc.2005.08.02715. Zhang, L, Zhou, W, Li, D-H: A descent modified Polak-Ribiére-Polyak conjugate gradient method and its global
convergence. IMA J Numer Anal. 26, 629–640 (2006). doi:10.1093/imanum/drl01616. Yuan, G: modified nonlinear conjugate gradient method with sufficient descent property for large-scale optimization
problems. Optim Lett. 3, 11–21 (2009). doi:10.1007/s11590-008-0086-517. Gaohang, Yu, Lutai, Guan, Wufan, Chen: Spectral conjugate gradient methods with sufficient descent property for large-
pp. 37–86. North-Holland, Amsterdam (1970)20. Dai, Y, Ni, Q: Testing different conjugate gradient methods for large-scale unconstrained optimization. J Comput Math.
213, 11–320 (2003)21. Griewank, A: On automatic differentiation. In: Iri M, Tannabe K (eds.) Mathematical Programming: Recent Developments
and Applications. pp. 84–108. Kluwer Academic Publishers, Dordrecht (1989)22. Dolan, ED, Moré, JJ: Benchmarking optimization software with performance profiles. Math Program. 91, 201–213 (2001)
doi:10.1186/1029-242X-2011-57Cite this article as: Liu and Wang: Modified nonlinear conjugate gradient method with sufficient descentcondition for unconstrained optimization. Journal of Inequalities and Applications 2011 2011:57.
Liu and Wang Journal of Inequalities and Applications 2011, 2011:57http://www.journalofinequalitiesandapplications.com/content/2011/1/57