Distilling Reasoning Capabilities into Smaller Language Models
Kumar Shridhar ∗ Alessandro Stolfo Mrinmaya Sachan Department of Computer Science, ETH Zürich {shkumar, Equal contribution;
Abstract
Step-by-step reasoning approaches like chain of thought (CoT) have proved to be very effective in inducing reasoning capabilities in large language models. However, the success of the CoT approach is fundamentally tied to the model size, and billion parameter-scale models are often needed to get CoT to work. In this paper, we propose a knowledge distillation approach that leverages the step-by-step CoT reasoning capabilities of larger models and distills these abilities into smaller models.
中文速览
链式思维（Chain-of-Thought, CoT）推理虽能显著提升大语言模型的表现，但这种能力几乎只在千亿参数级别的超大模型上才能涌现，计算成本极高。为此，作者提出了一种知识蒸馏框架：先用大模型生成带有推理步骤的解题标注，再用这些标注来微调小模型，从而把大模型的推理能力"灌"进参数量小得多的学生模型。其中最关键的创新是"苏格拉底式CoT"（Socratic CoT）——不是简单复制CoT的连续推理步骤，而是将原问题分解为一系列子问题，训练一个"分解器"负责提问、一个"求解器"负责逐步回答，两者协同完成复杂推理。在GSM8K、StrategyQA、SVAMP三个多步推理基准上，该方法让小模型的性能相比基线提升超过70%，甚至让GPT-2 large这样的小模型在某些场景下超越了体量大10倍的GPT-3 6B，为在资源受限环境下部署具备推理能力的语言模型提供了切实可行的路径。
原文 arXiv:2212.00193;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2212.00193v2