1. Non-asymptotic World Convergence Evaluation of BFGS with the Armijo-Wolfe Line Search
Authors: Qiujiang Jin, Ruichen Jiang, Aryan Mokhtari
Summary: On this paper, we set up the primary specific and non-asymptotic international convergence evaluation of the BFGS technique when deployed with an inexact line search scheme that satisfies the Armijo-Wolfe situations. We present that BFGS achieves a worldwide convergence fee of (1−1κ)ok for μ-strongly convex features with L-Lipschitz gradients, the place κ=Lμ denotes the situation quantity. Moreover, if the target perform’s Hessian is Lipschitz, BFGS with the Armijo-Wolfe line search achieves a linear convergence fee solely decided by the road search parameters and unbiased of the situation quantity. These outcomes maintain for any preliminary level x0 and any symmetric constructive particular preliminary Hessian approximation matrix B0, though the selection of B0 impacts the iteration rely required to achieve these charges. Particularly, we present that for B0=LI, the speed of O((1−1κ)ok) seems from the primary iteration, whereas for B0=μI, it takes dlogκ iterations. Conversely, the situation number-independent linear convergence fee for B0=LI happens after O(κ(d+Mf(x0)−f(x∗)√μ3/2)) iterations, whereas for B0=μI, it holds after O(Mf(x0)−f(x∗)√μ3/2(dlogκ+κ)) iterations. Right here, d denotes the dimension of the issue, M is the Lipschitz parameter of the Hessian, and x∗ denotes the optimum answer. We additional leverage these international linear convergence outcomes to characterize the general iteration complexity of BFGS when deployed with the Armijo-Wolfe line search.
