With exact line search, our method reduces to a nonlinear version of the hestenesstiefel conjugate gradient scheme. Wei, new line search methods for unconstrained optimization, journal of the korean statistical society, 382009, pp. A modi ed of nonmonotone spectral conjugate gradient method. On the subspace minimization conjugate gradient method.
In this paper, we propose a nonmonotone line search combining with the search direction g. An introduction to the conjugate gradient method without the. The method incorporates the modified bfgs secant equation in an effort to include the second order information of the objective function. The additional orthogonality reduces the gramschmidt process to threeterm recursion. However, for some illconditioned problems, orthogonality is quickly lost due to rounding errors, and convergence is much slower than expected. The conjugate gradient method can be applied to an arbitrary nbym matrix by applying it to normal equations a t a and righthand side vector a t b, since a t a is a symmetric positivesemidefinite matrix for any a. The spectral projected gradient spg method birgin, mart nez, and raydan2000,2001. A nonmonotone hybrid conjugate gradient method is proposed, in which the technique of the nonmonotone wolfe line search is used. An iterative conjugate gradient regularization method for. A riemannian conjugate gradient method for optimization on. A nonmonotone conjugate gradient algorithm for unconstrained optimization article in journal of systems science and complexity 152 january 2002 with 60 reads how we measure reads.
A nonmonotone prp conjugate gradient method for solving. With exact line search, our method reduces to a nonlinear ver. Nonmonotone conjugate gradient method nonmonotone line search global convergence unconstrained optimization. We applied the spectral steplength to the entire conjugate gradient direction rather than the negative gradient. Therefore, there existed a wide space for variations and extensions of the bb original method. The global convergence of the given method will be established under suitable condi. Although the steepest descent method converges, it is inef. Li and yang journal of inequalities and applications a nonmonotone hybrid conjugate gradient method for unconstrained optimization wenyu li 0 yueting yang 0 0 school of mathematics and statistics, beihua university, jilin street no.
Nonlinear conjugate gradient methods, unconstrained optimization, nonlinear. Both of them satisfy the ringwirth nonexpansive condition, which is fundamental for convergence analysis of riemannian conjugate gradient methods, and one of them is also. A new nonlinear conjugate gradient method and an associated implementation, based on an inexact line search, are proposed and analyzed. A comparative study of non linear conjugate gradient methods. Nov 10, 2016 in this paper we propose a new riemannian conjugate gradient method for optimization on the stiefel manifold. A class of nonmonotone conjugate gradient methods for. An introduction to the conjugate gradient method without the agonizing pain jonathan richard shewchuk march 7, 1994 cmucs94125 school of computer science carnegie mellon university pittsburgh, pa 152 abstract the conjugategradient method is themost prominent iterativemethod for solvingsparse systems of linear equations. The spectral gradient method has been successfully extended in for solving square nonlinear systems of equations using grippos nonmonotone line search technique. Conjugate gradient method employs vectors that are aorthogonal or conjugate details of the derivation of the method are omitted r 1 r 0 0 t j 0 t d i ad. This new line search technique is based on a relaxation of the strong wolfe conditions and it allows to accept larger steps. Dais nonmonotone conjugate gradient method is generalized to the riemannian case and global convergence of the new algorithm is established under standard assumptions.
An iterative conjugate gradient regularization method for image restoration. This makes them easy to implement and they do not require much storage. Numerical results on a variety of lowrank test problems demonstrate the effectiveness of the new method. This paper is devoted to a riemannian conjugate gradient method for solving problem 1. In this paper, we combined a new nonmonotone techniques with the spectral conjugate gradient method to obtain a more e. A hybrid conjugate gradient method for optimization problems. In this paper, combining the nonmonotone and monotone line search,a spectral conjugate gradient methods are used in this paper. A limited memory version of the nonlinear conjugate gradient method is developed. On the convergence of a new conjugate gradient algorithm. It is well known that in the euclidean space, the conjugate gradient method generally outperforms the steepest descent method for its faster convergence and is more suitable than secondorder methods such as newtons method, quasinewton. Krylov subspaces and conjugate gradients c 2006 gilbert strang 6. Nonmonotone spectral projected gradient methods on convex sets. A new conjugate gradient method with guaranteed descent and. Preliminary numerical results show that this method is very ef.
Numerical experiments show that the nonmonotone polakribiere method and hestenesstiefel method in this nonmonotone conjugate gradient class are. Image restoration is an illposed inverse problem, which has been introduced the regularization method to suppress over. In this paper conjugate gradient methods with nonmonotone line search technique are introduced. The convergence of conjugate gradient method with nonmonotone. A nonmonotone line search technique for newtons method. A scaled conjugate gradient method based on new bfgs. The steepest descent and the conjugate gradient methods both do not require estimation of the hessian. A nonmonotone line search method for regression analysis. Both of these methods have a qlinear rate of convergence. A new nonmonotone spectral conjugate gradient method for.
A nonmonotone hybrid conjugate gradient method for. The new residual is orthogonal to the whole space not only to one residual vector in the previous step. Request pdf the convergence of conjugate gradient method with nonmonotone line search the conjugate gradient method is a useful and powerful approach for. The cg method has the simplicity and the very low memory requirement and the prp method is one of the most effective conjugate gradient methods. We then of n are being very large, say, n 106 or n 107. P may come from incomplete lu, or a few steps of a multigrid iteration, or. Under mild assumptions, we prove the global convergence and linear convergence rate of the method. A hybrid method of the polakribierepolyak prp method and the weiyaoliu wyl method is proposed for unconstrained optimization pro blems, which possesses the following properties. This lemma shows the advantage of the conjugate gradient method over the gradient method. In this paper, we introduce a class of nonmonotone conjugate gradient methods, which include the wellknown polakribire method and hestenesstiefel method as special cases. In theory, the successive gradients generated by the conjugate gradient method applied to a quadratic should be orthogonal. The spectral gradient and conjugate gradient methods are a class of methods that can suitably cope with largescale settings.
In this paper, we provide and analyze a new scaled conjugate gradient method and its performance, based on the modified secant equation of the broydenfletchergoldfarbshanno bfgs method and on a new modified nonmonotone line search technique. Nonmonotone conjugate gradient methods for optimization. We introduce two novel vector transports associated with the retraction constructed by the cayley transform. When we write p 1, we never intend that an inverse will be explicitly computed. A new conjugate gradient method with guaranteed descent. In particular, a scaled version of the conjugate gradient method, suggested by perry 2, 9, which employ the spectral steplength of barzilai and borwein 1, 10, was. In this paper, we introduce a class of nonmonotone conjugate gradient methods, which include the wellknown polakribiere method and hestenesstiefel method as special cases. By making use of the moreauyosida regularization, a nonmonotone line search technique of 48 and a new secant equation of 43 derived by the authors earlier, we present a modified prp conjugate.
A nonmonotone conjugate gradient algorithm for unconstrained. Request pdf the convergence of conjugate gradient method with nonmonotone line search the conjugate gradient method is a useful and powerful approach for solving largescale minimization problems. The result is conjugate gradient on the normal equations cgnr. A modified polakribierepolyak conjugate gradient algorithm. On the subspace minimization conjugate gradient method yuhong dai center for optimization and applications amss, chinese academy of sciences discussing with y. A scaled conjugate gradient method based on new bfgs secant. A modified spectral conjugate gradient methods with the new. Optimization online a riemannian conjugate gradient method.
102 1328 979 182 119 367 474 1475 1063 899 418 1281 1374 312 1086 486 660 1352 657 1583 72 328 1201 324 1541 1366 1016 285 1298 704 581 167 824 233 212 1391 592 176 331 1083