Application of spectral conjugate gradient methods for solving unconstrained optimization problems
DOI:
https://doi.org/10.11121/ijocta.01.2020.00859Keywords:
Sufficient descent property, Exact line search, Regression analysis, Spectral CGAbstract
Conjugate gradient (CG) methods are among the most efficient numerical methods for solving unconstrained optimization problems. This is due to their simplicty and less computational cost in solving large-scale nonlinear problems. In this paper, we proposed some spectral CG methods using the classical CG search direction. The proposed methods are applied to real-life problems in regression analysis. Their convergence proof was establised under exact line search. Numerical results has shown that the proposed methods are efficient and promising.
Downloads
References
Andrei, N. (2008). An unconstrained optimization test functions collection, Adv. Modell. Optim. 10, 147-161.
Barzilai, J. & Borwein, J.M. (1988). Two-point step size gradient methods, IMA J Numer Anal. 8, 141-148.
Birgin, E.G. & Martinez, J. M. (2011). A spectral conjugate gradient method for unconstrained optimization, Appl. Math. Optim. 43 (2), 117-128.
Yakubu, U.A. Mamat, M. Mohamad, A.M. Sukono, A.M. & Rivaie, M. (2018a). Secant free condition of a spectral PRP conjugate gradient method. International Journal of Engineering & Technology, 7 (3.28), 325 – 328.
Yakubu, U.A. Mamat, M. Mohamad, A.M. Sukono, A.M. & Rivaie, M. (2018b). Modification on spectral conjugate gradient method for unconstrained optimization. International Journal of Engineering & Technology, 7 (3.28), 307 – 311.
Yakubu, U.A. Mamat, M. Mohamad, A.M. Rivaie, M. & B.Y. Rabi’u. (2018d). Secant free condition of a spectral WYL and its global convergence properties. Far East Journal of Mathematical science, 12, 1889 – 1902.
Yakubu, U.A. Mamat, M. Mohamad, A.M. Rivaie, M. & J. Sabi’u. (2018e). A recent modification on Dai-Liao conjugate gradient method for solving symmetric nonlinear equations. Far East Journal of Mathematical science, 12, 1961 – 1974.
Zull, N. Rivaie, M. Mamat, M. Salleh, Z. & Amani, Z. (2015). Global convergence of a Spectral conjugate gradient by using strong Wolfe line search, Appl. Math. Sci. 63, 3105-3117.
Hu, C. & Wan, Z. (2013). An Extended Spectral Conjugate Gradient Method for unconstrained optimization problems, British Journal of Mathematics & Computer Science, 3, 86-98.
X. Wu. (2015). A new spectral Polak- Ribière -Polak conjugate gradient method, ScienceAsia. 41, 345-349.
Sulaiman, I. M., Sukono, Sudradjat, S., and Mamat, M. (2019). New class of hybrid conjugate gradient coefficients with guaranteed descent and efficient line search. IOP Conf. Ser.: Mater. Sci. Eng. 621(012021), 1-7.
Mathematical Sciences Issue 65-68, 3307-3319.
W. W. Hager & H. Zhang. (2006). A survey of nonlinear conjugate gradient methods, Pacific Journal of Optimization.. 2 (1), 35-58.
Du, X. & Liu, J. (2011). Global convergence of a spectral HS conjugate gradient method, Procedia Engineering. 15, 1487 – 1492.
Raydan, M. (1997). The Barzilai and J.M. Borwein gradient methods for the large scale unconstrained minimization in extreme problems, SIAM. J. Optim. 7 (1), 26-33.
Batu, T., Dasgupta, S., Kumar, R., & Rubinfeld, R. (2005). The complexity of approximating the entropy. SIAM Journal on Computing, 35(1): 132–150.
Motulsky, H., & Christopoulos, A. (2004). Fitting models to biological data using linear and nonlinear regression. Oxford University Press, New York.
Shirin, S., Mahmudul. H., & Laek, S.A. (2015). Age-Structured Population Projection of Bangladesh by Using a Partial Differential Model with Quadratic Polynomial Curve Fitting. Journal of Applied Sciences, 5: 542-551.
Zoutendijk, G. “Nonlinear programming, computational methods, in J Abadie (ED)”, Integer and Nonlinear Programming, North-Holland, Amsterdam, (1970), 37-86.
Yakubu, U.A. (2019). Enhancing spectral conjugate gradient method for solving unconstrained optimization problems. PhD Thesis. University Sultan Zainal Abidin Kuala Terengganu, Malaysia.
Yakubu, U.A. Mamat, M. Mohamad, A.M. Puspa, L.G. & Rivaie, M. (2018c). Secant free condition of a spectral Hestenes-Stiefel conjugate gradient method and its sufficient descent properties. International Journal of Engineering & Technology, 7 (3.28), 312 – 315.
K.E. Hillstrom. (1977). A simulation test approach to the evaluation of the nonlinear optimization algorithm. Journal ACM Trans. Mathematics Software. 3 (4), 305-315.
Dolan, E. & More, J.J. (2002). Benchmarking optimization software with performance profile, Math. Prog. Vol. 91, 201-213.
Z. Wei, S. Yao, and L. Liu. (2006). The convergence properties of some new conjugate gradient methods, Applied Mathematics and Computation, Vol. 183(2), pp. 1341–1350.
E. Polak & G. Ribiere. (1969). Note sur la convergence de directions conjug´ees, Rev. Francaise Informat Recherche Opertionelle, 3e ann´ee, Vol 16, pp. 35–43.
Sulaiman, I. M. Mamat, M. Abashar, A. Rivaie M. (2015). The global convergence properties of an improved conjugate gradient method Applied Mathematical Science; Vol 9(38), pp. 1857-1868.
Powell, M.J.D. (1977). Restart procedures for the conjugate gradient method, Math. Program. 12, 241-254.
Sulaiman, I. M., Mamat, M., Abashar, A., Zabidin, S. (2015): A Modified Nonlinear Conjugate Gradient Method for Unconstrained Optimization, Applied Mathematical Sciences Vol. 9(54), pp. 2671-2682.
Aini, N. Rivaie, M. Mamat, M. and Sulaiman, I. M. (2019). A Hybrid of Quasi-Newton Method with CG Method for Unconstrained Optimization. J. Phys.: Conf. Ser. 1366(012079), 1-10.
Downloads
Published
How to Cite
Issue
Section
License
Articles published in IJOCTA are made freely available online immediately upon publication, without subscription barriers to access. All articles published in this journal are licensed under the Creative Commons Attribution 4.0 International License (click here to read the full-text legal code). This broad license was developed to facilitate open access to, and free use of, original works of all types. Applying this standard license to your work will ensure your right to make your work freely and openly available.
Under the Creative Commons Attribution 4.0 International License, authors retain ownership of the copyright for their article, but authors allow anyone to download, reuse, reprint, modify, distribute, and/or copy articles in IJOCTA, so long as the original authors and source are credited.
The readers are free to:
- Share — copy and redistribute the material in any medium or format
- Adapt — remix, transform, and build upon the material
- for any purpose, even commercially.
- The licensor cannot revoke these freedoms as long as you follow the license terms.
under the following terms:
- Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
This work is licensed under a Creative Commons Attribution 4.0 International License.
