Robust Task Clustering for Deep and Diverse Multi-Task and Few-Shot Learning
Mo Yu Xiaoxiao Guo∗ Jinfeng Yi∗ Shiyu Chang Saloni Potdar Gerald Tesauro Haoyu Wang Bowen Zhou AI Foundations – Learning, IBM Research IBM T. J. Watson Research Center, Yorktown Heights, NY 10598 Equal contributions. Corresponding authors:
Abstract
We investigate task clustering for deep learning-based multi-task and few-shot learning in the settings with large numbers of tasks. Our method measures task similarities using cross-task transfer performance matrix. Although this matrix provides us critical information regarding similarities between tasks, the uncertain task-pairs, i.e., the ones with extremely asymmetric transfer scores, may collectively mislead clustering algorithms to output an inaccurate task-partition. Moreover, when the number of tasks is large, generating the full transfer performance matrix can be very time consuming. To overcome these limitations, we propose a novel task clustering algorithm to estimate the similarity matrix based on the theory of matrix completion. The proposed algorithm can work on partially-observed similarity matrices based on only sampled task-pairs with reliable scores, ensuring its efficiency and robustness. Our theoretical analysis shows that under mild assumptions, the reconstructed matrix perfectly matches the underlying “true” similarity matrix with an overwhelming probability. The final task partition is computed by applying an efficient spectral clustering algorithm to the re
中文速览
当面对数百甚至上千个文本分类任务时，如何自动把"相互有益"的任务归为一组、避免不相关任务互相拖累，是多任务学习（multi-task learning）和少样本学习（few-shot learning）的核心难题。研究者提出用"跨任务迁移性能矩阵"来衡量任务间的相似度——即把一个任务训练好的特征表示迁移到另一个任务上，看准确率有多高——但直接计算所有任务对需要天文数量级的实验，且部分评分因数据噪声而不可靠。为此，他们引入矩阵补全（matrix completion）理论：只随机采样少量可信任务对来构建一个稀疏的相似度矩阵，再用鲁棒矩阵补全算法还原完整矩阵，最后用谱聚类（spectral clustering）得到任务分组；理论证明只需 O(n log²n) 个样本即可以极大概率完美还原真实相似度矩阵，远优于暴力计算的 O(n²)。在情感分类和对话意图分类的实验中，该任务聚类方法显著提升了多任务学习和少样本学习的性能，为大规模异构任务场景提供了兼顾效率与鲁棒性的实用解决方案。
原文 arXiv:1708.07918;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1708.07918v2