One Size Fits All: Can We Train One Denoiser for All Noise Levels?
Abhiram Gnansambandam Stanley H. Chan
Abstract
When training an estimator such as a neural network for tasks like image denoising, it is often preferred to train one estimator and apply it to all noise levels. The de facto training protocol to achieve this goal is to train the estimator with noisy samples whose noise levels are uniformly distributed across the range of interest. However, why should we allocate the samples uniformly? Can we have more training samples that are less noisy, and fewer samples that are more noisy? What is the optimal distribution? How do we obtain such a distribution? The goal of this paper is to address this training sample distribution problem from a minimax risk optimization perspective. We derive a dual ascent algorithm to determine the optimal sampling distribution of which the convergence is guaranteed as long as the set of admissible estimators is closed and convex. For estimators with non-convex admissible sets such as deep neural networks, our dual formulation converges to a solution of the convex relaxation. We discuss how the algorithm can be implemented in practice. We evaluate the algorithm on linear estimators and deep networks.
中文速览
训练一个能应对所有噪声等级的"万能去噪器"时，惯常做法是在训练集里对噪声等级做均匀采样，但这往往导致低噪声端性能明显变差、整体表现不均衡。本文将这个"应该怎么分配训练样本"的问题建模为一个极小极大（minimax）风险优化：在最小化整体风险的同时，约束任意噪声等级下与"针对该等级单独训练的最优器"之间的差距不超过某一阈值。通过分析该问题的对偶形式，作者推导出一个对偶上升（dual ascent）算法，在估计器的可行集封闭且凸时可证明收敛，对深度神经网络等非凸情形则收敛至凸松弛解；算法的每一步本质上就是迭代更新训练样本的噪声等级分布，实现简便。在线性估计器和深度网络上的实验均验证了该方法能显著改善去噪性能在全噪声范围内的均衡性，为"一模型覆盖所有噪声等级"这一实用目标提供了有理论保障的最优采样方案。
原文 arXiv:2005.09627;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2005.09627v3