Learning Efficient Sparse and Low Rank Models
P. Sprechmann, A. M. Bronstein, and G. Sapiro∗ P. Sprechmann and G. Sapiro are with the Department of Electrical and Computer Engineering, Duke University, Durham 27708, USA. Email: M. Bronsteind is with School of Electrical Engineering, Tel Aviv University, Tel Aviv 69978, Israel.Email: partially supported by NSF, ONR, NGA, DARPA, AFOSR, ARO, and BSF.
Abstract
Parsimony, including sparsity and low rank, has been shown to successfully model data in numerous machine learning and signal processing tasks. Traditionally, such modeling approaches rely on an iterative algorithm that minimizes an objective function with parsimony-promoting terms. The inherently sequential structure and data-dependent complexity and latency of iterative optimization constitute a major limitation in many applications requiring real-time performance or involving large-scale data. Another limitation encountered by these modeling techniques is the difficulty of their inclusion in discriminative learning scenarios. In this work, we propose to move the emphasis from the model to the pursuit algorithm, and develop a process-centric view of parsimonious modeling, in which a learned deterministic fixed-complexity pursuit process is used in lieu of iterative optimization. We show a principled way to construct learnable pursuit process architectures for structured sparse and robust low rank models, derived from the iteration of proximal descent algorithms. These architectures learn to approximate the exact parsimonious representation at a fraction of the complexity of the s
中文速览
稀疏性与低秩分解等"简约表示"方法在信号处理和机器学习中大放异彩，但传统做法依赖迭代优化算法，不仅速度慢、延迟高，还难以融入有监督的判别学习框架。这篇论文提出了一套"以过程为中心"的简约建模框架：不再死守迭代求解器，而是从近端梯度下降算法的迭代结构出发，直接设计出可学习的固定计算量神经网络追踪架构，让网络"学会"近似稀疏编码、分组稀疏、鲁棒主成分分析（Robust PCA）和非负矩阵分解（NMF）等多种模型的输出。得益于端到端训练，这些编码器既能以极低复杂度快速逼近精确优化结果，又能自然地嵌入判别任务（如图像分类、说话人识别）中，在多个图像和音频处理基准上达到或超越原有迭代方法的精度，同时实现数个数量级的提速、甚至达到快于实时的性能。这项工作的意义在于，它将稀疏/低秩模型从"求解优化问题"的范式解放出来，为深度学习与经典信号处理的融合提供了一条原则性的桥梁。
原文 arXiv:1212.3631;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1212.3631v1