An Approach for Analyzing the Global Rate of Convergence of Quasi-Newton and Truncated-Newton Methods

详细信息    查看全文

• 作者：T. L. Jensen ; M. Diehl
• 关键词：Quasi/truncated ; Newton methods ; First ; order methods ; Complexity analysis
• 刊名：Journal of Optimization Theory and Applications
• 出版年：2017
• 出版时间：January 2017
• 年：2017
• 卷：172
• 期：1
• 页码：206-221
• 全文大小：
• 刊物主题：Calculus of Variations and Optimal Control; Optimization; Optimization; Theory of Computation; Applications of Mathematics; Engineering, general; Operation Research/Decision Theory;
• 出版者：Springer US
• ISSN：1573-2878
• 卷排序：172

文摘

Quasi-Newton and truncated-Newton methods are popular methods in optimization and are traditionally seen as useful alternatives to the gradient and Newton methods. Throughout the literature, results are found that link quasi-Newton methods to certain first-order methods under various assumptions. We offer a simple proof to show that a range of quasi-Newton methods are first-order methods in the definition of Nesterov. Further, we define a class of generalized first-order methods and show that the truncated-Newton method is a generalized first-order method and that first-order methods and generalized first-order methods share the same worst-case convergence rates. Further, we extend the complexity analysis for smooth strongly convex problems to finite dimensions. An implication of these results is that in a worst-case scenario, the local superlinear or faster convergence rates of quasi-Newton and truncated-Newton methods cannot be effective unless the number of iterations exceeds half the size of the problem dimension.