Non-convex Optimization for Machine Learning
Prateek Jain Microsoft Research India Purushottam Kar IIT Kanpur
Abstract
A vast majority of machine learning algorithms train their models and perform inference by solving optimization problems. In order to capture the learning and prediction problems accurately, structural constraints such as sparsity or low rank are frequently imposed or else the objective itself is designed to be a non-convex function. This is especially true of algorithms that operate in high-dimensional spaces or that train non-linear models such as tensor models and deep networks.
中文速览
非凸优化（non-convex optimization）长期被视为"不可解"的难题，但机器学习和信号处理中的稀疏恢复、矩阵补全、鲁棒回归等任务天然带有额外结构，使得直接求解非凸问题成为可能。这部专著系统梳理了投影梯度下降（projected gradient descent）、交替最小化（alternating minimization）、EM算法和随机非凸优化这几类核心算法，深入分析它们在上述结构化问题上的收敛性与正确性保证。实验与理论双双表明，这些直接方法往往比先把非凸问题松弛成凸问题再求解的传统路线快一个数量级，同时精度相当。对于想在机器学习和信号处理领域用好非凸方法、又希望理解"为什么有效"的研究者和学生，这部专著提供了从基础工具到具体应用的完整脉络。
原文 arXiv:1712.07897;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1712.07897v1