Block Pruning For Faster Transformers
François Lagunas, Ella Charlaix, Victor Sanh, Alexander M. Rush Hugging Face {francois, ella, victor,
Abstract
Pre-training has improved model accuracy for both classification and generation tasks at the cost of introducing much larger and slower models. Pruning methods have proven to be an effective way of reducing model size, whereas distillation methods are proven for speeding up inference. We introduce a block pruning approach targeting both small and fast models. Our approach extends structured methods by considering blocks of any size and integrates this structure into the movement pruning paradigm for fine-tuning. We find that this approach learns to prune out full components of the underlying model, such as attention heads. Experiments consider classification and generation tasks, yielding among other results a pruned model that is a 2.4x faster, 74% smaller BERT on SQuAD v1, with a 1% drop on F1, competitive both with distilled models in speed and pruned models in size.
中文速览
预训练模型（如BERT）虽然精度高，但体积庞大、推理缓慢，现有剪枝方法能压缩参数量却难以真正提速，蒸馏方法能加速但压缩率又不够理想。为此，研究者提出"块移动剪枝"（Block Movement Pruning）：把模型权重矩阵划分为固定大小的块，在微调阶段以块为单位进行移动剪枝，使稀疏结构天然契合GPU的密集矩阵运算需求。实验发现一个惊喜现象：即便训练时使用的是小方块（32×32），模型会自动学会整块消除注意力头等完整组件，从而在不显式指定结构的情况下实现真正的结构化稀疏。最终在SQuAD v1.1上，该方法以仅1% F1下降为代价，获得了比原版BERT快2.4倍、参数量减少74%的模型，同时兼顾了蒸馏方法的推理速度和剪枝方法的压缩率，为在普通硬件上高效部署大型预训练模型提供了一条切实可行的新路径。
原文 arXiv:2109.04838;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2109.04838v1