BANet: A Blur-aware Attention Network for Dynamic Scene Deblurring
Fu-Jen Tsai∗, Yan-Tsung Peng∗, Chung-Chi Tsai, Yen-Yu Lin, Chia-Wen Lin, , *equal contribution Manuscript received August 16, 2021; revised May 17, 2022 and August 30, 2022; accepted October 13, 2022. Date of publication month day, 2022; date of current version month day, 2022. This work was funded in part by National Science and Technology Council (NSTC) under grants 110-2634-F-002-050, 111-2628-E-A49-025-MY3, 111-2221-E-004-010, and in part by Qualcomm Technologies, Inc., through a Taiwan University Research Collaboration Project, under Grant NAT-487844. The computational and storage resources for this project were provided in part by National Center for High-performance Computing (NCHC) of National Applied Research Laboratories (NARLabs), Hsinchu, Taiwan. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Wangmeng Zuo. (Corresponding author: Chia-Wen Lin)F.-J. Tsai is with the Department of Electrical Engineering, National Tsing Hua University, Hsinchu 300044, Taiwan. E-mail: Peng is with the Department of Computer Science, National Chengchi University, Taipei 116011, Taiwan. E-mail: Tsai is with Qualcomm Technologies, Inc., San Diego, CA 92121, USA. E-mail: Lin is with the Department of Computer Science, National Yang Ming Chiao Tung University, Hsinchu 300093, Taiwan. E-mail: Lin is with the Department of Electrical Engineering, National Tsing Hua University, Hsinchu 300044, Taiwan, and with the Electronic and Optoelectronic System Research Laboratories, Industrial Technology Research Institute, Hsinchu 310401, Taiwan. (e-mail:
Abstract
Image motion blur results from a combination of object motions and camera shakes, and such blurring effect is generally directional and non-uniform. Previous research attempted to solve non-uniform blurs using self-recurrent multi-scale, multi-patch, or multi-temporal architectures with self-attention to obtain decent results. However, using self-recurrent frameworks typically leads to a longer inference time, while inter-pixel or inter-channel self-attention may cause excessive memory usage. This paper proposes a Blur-aware Attention Network (BANet), that accomplishes accurate and efficient deblurring via a single forward pass. Our BANet utilizes region-based self-attention with multi-kernel strip pooling to disentangle blur patterns of different magnitudes and orientations and cascaded parallel dilated convolution to aggregate multi-scale content features. Extensive experimental results on the GoPro and RealBlur benchmarks demonstrate that the proposed BANet performs favorably against the state-of-the-arts in blurred image restoration and can provide deblurred results in real-time.
中文速览
运动模糊（motion blur）普遍存在于手机拍照和监控视频中，其方向各异、强度不均，极难一次性去除干净。现有深度学习方法要么反复迭代导致推理太慢，要么用像素级自注意力（self-attention）导致显存爆炸，难以实时运行。为此，作者提出了模糊感知注意力网络（Blur-aware Attention Network，BANet），其核心是一种"区域条形池化"注意力模块——用水平和垂直多尺度均值池化来模拟运动模糊本身的方向性和平均化特性，从而以极低的显存开销捕捉不同方向和强度的模糊分布，再配合级联并行扩张卷积恢复清晰内容；整个过程只需单次前向传播，无需任何迭代。在GoPro和RealBlur两个标准测试集上，BANet的去模糊质量达到当前最优水平，同时能以实时速度处理高清视频帧，真正实现了精度与速度的双赢，对人脸检测、文字识别等下游视觉任务具有重要的实用价值。
原文 arXiv:2101.07518;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2101.07518v4