Large Margin Distribution Machine
Teng Zhang Zhi-Hua Zhou National Key Laboratory for Novel Software Technology Nanjing University, Nanjing 210023, China
Abstract
Support vector machine (SVM) has been one of the most popular learning algorithms, with the central idea of maximizing the minimum margin, i.e., the smallest distance from the instances to the classification boundary. Recent theoretical results, however, disclosed that maximizing the minimum margin does not necessarily lead to better generalization performances, and instead, the margin distribution has been proven to be more crucial. In this paper, we propose the Large margin Distribution Machine (LDM), which tries to achieve a better generalization performance by optimizing the margin distribution. We characterize the margin distribution by the first- and second-order statistics, i.e., the margin mean and variance. The LDM is a general learning approach which can be used in any place where SVM can be applied, and its superiority is verified both theoretically and empirically in this paper.
中文速览
传统支持向量机（SVM）的核心思想是最大化"最小间隔"，但近年理论研究表明，最小间隔并不能充分决定模型的泛化性能，真正关键的是整体的间隔分布（margin distribution）。为此，作者提出了大间隔分布机（Large margin Distribution Machine，LDM），通过同时最大化间隔均值、最小化间隔方差来优化整体间隔分布，而非只盯住最小那个间隔。在算法实现上，针对核方法场景设计了对偶坐标下降求解器，针对大规模线性场景设计了平均随机梯度下降（ASGD）方法，并从理论上给出了基于留一法的误差期望上界。在20个常规数据集和12个大规模数据集上的实验表明，LDM全面优于标准SVM及多种当前最先进方法，从实践层面有力证明了间隔分布比最小间隔对于SVM类学习方法更为重要，为改进这一经典算法提供了新思路。
原文 arXiv:1311.0989;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1311.0989v2