A Homogeneous Second-Order Descent Method for Nonconvex Optimization
Chuwen Zhang Thanks: This research is partially supported by the National Natural Science Foundation of China (NSFC) [Grant NSFC-72150001, 72225009, 11831002] and the Natural Science Foundation of Shanghai [23ZR1445900]. Affiliation: School of Information Management and Engineering Shanghai University of Finance and Economics Dongdong Ge Affiliation: Antai College of Economics and Management Shanghai Jiao Tong University Chang He Affiliation: School of Information Management and Engineering Shanghai University of Finance and Economics Yuntian Jiang Affiliation: School of Information Management and Engineering Shanghai University of Finance and Economics Chenyu Xue Affiliation: School of Information Management and Engineering Shanghai University of Finance and Economics Bo Jiang Affiliation: School of Information Management and Engineering Shanghai University of Finance and Economics Yinyu Ye Affiliation: Department of Management Science and Engineering, Stanford University
Abstract
In this paper, we introduce a Homogeneous Second-Order Descent Method (HSODM) motivated from the homogenization trick in quadratic programming. The merit of homogenization is that only the leftmost eigenvector of a gradient-Hessian integrated matrix is computed at each iteration. Therefore, the algorithm is a single-loop method that does not need to switch to other sophisticated algorithms and is easy to implement. We show that HSODM has a global convergence rate of $O(\epsilon^{-3/2})$ to find an $\epsilon$ -approximate second-order stationary point, and has a local quadratic convergence rate under the standard assumptions. The numerical results demonstrate the advantage of the proposed method over other second-order methods.
原文 arXiv:2211.08212;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2211.08212v7