Domain Adaptation: Learning Bounds and Algorithms
Yishay Mansour Google Research and Tel Aviv Univ.、Mehryar Mohri Courant Institute and Google Research、Afshin Rostamizadeh Courant Institute New York University
Abstract
This paper addresses the general problem of domain adaptation which arises in a variety of applications where the distribution of the labeled sample available somewhat differs from that of the test data. Building on previous work by ? (?), we introduce a novel distance between distributions, discrepancy distance, that is tailored to adaptation problems with arbitrary loss functions. We give Rademacher complexity bounds for estimating the discrepancy distance from finite samples for different loss functions. Using this distance, we derive novel generalization bounds for domain adaptation for a wide family of loss functions. We also present a series of novel adaptation bounds for large classes of regularization-based algorithms, including support vector machines and kernel ridge regression based on the empirical discrepancy. This motivates our analysis of the problem of minimizing the empirical discrepancy for various loss functions for which we also give novel algorithms. We report the results of preliminary experiments that demonstrate the benefits of our discrepancy minimization algorithms for domain adaptation.
中文速览
领域自适应（domain adaptation）问题的核心挑战在于：训练数据和测试数据来自不同分布，如何让模型在目标域上仍表现良好。本文提出了一种新的分布差异度量——"差异距离"（discrepancy distance），它推广了前人针对分类任务设计的 $d_A$ 距离，可适用于回归等更广泛的损失函数场景，并通过 Rademacher 复杂度给出了从有限样本估计该距离的数据依赖型上界。基于这一距离，作者推导出一系列新的泛化界，覆盖支持向量机（SVM）和核岭回归（KRR）等正则化算法，并将界中的关键项直接与源域和目标域之间的经验差异距离挂钩，从而自然引出"最小化差异距离"的算法思路——对0-1损失可转化为线性规划（LP），对平方损失可转化为半正定规划（SDP）求解。初步实验验证了这类差异最小化算法在领域自适应任务中的实际效果，为统一理解和改进跨域学习提供了坚实的理论与算法基础。
原文 arXiv:0902.3430;中英对照 + 大白话阅读 https://aha.fim.ai/paper/0902.3430v3