Reasoning Like Program Executors
Xinyu Pi◆◆\lozenge , Qian Liu§∗, Bei Chen†, Morteza Ziyadi ♡♡\heartsuit, Zeqi Lin† Qiang Fu†, Yan Gao†, Jian-Guang Lou†, Weizhu Chen♡♡\heartsuit ◆◆\lozengeUniversity of Illinois Urbana-Champaign, Urbana, USA; §Sea AI Lab, Singapore †Microsoft Research Asia, Beijing, China; ♡♡\heartsuitMicrosoft Azure AI, Redmond, WA, USA {beichen, morteza.ziyadi, zeqi.lin, qifu, yan.gao, jlou, The first two authors contributed equally. Work done during internship at Microsoft Research Asia.
Abstract
Reasoning over natural language is a long-standing goal for the research community. However, studies have shown that existing language models are inadequate in reasoning. To address the issue, we present PoEt, a novel reasoning pre-training paradigm. Through pre-training language models with programs and their execution results, PoEt empowers language models to harvest the reasoning knowledge possessed by program executors via a data-driven approach. PoEt is conceptually simple and can be instantiated by different kinds of program executors. In this paper, we showcase two simple instances PoEt-Math and PoEt-Logic, in addition to a complex instance, PoEt-SQL. Experimental results on six benchmarks demonstrate that PoEt can significantly boost model performance in natural language reasoning, such as numerical reasoning, logical reasoning, and multi-hop reasoning. PoEt opens a new gate on reasoning-enhancement pre-training, and we hope our analysis would shed light on the future research of reasoning like program executors.
中文速览
大量自然语言推理任务对现有语言模型来说依然很困难，根本原因在于预训练阶段几乎没有让模型学过严格的推理逻辑。为此，研究者提出了 PoEt（Program Executor pre-training，程序执行器预训练）这一新范式：用数学计算器、逻辑求解器、SQL数据库引擎等程序执行器来充当"老师"，让语言模型在预训练阶段大量练习"给定程序和上下文、预测执行结果"的任务，从而把这些符号系统中蕴藏的推理知识内化到模型参数里。相比依赖嘈杂自然语言语料的现有方法，程序可以按照语法规则任意大规模合成，质量有保障、成本极低。在六个推理类下游基准上的实验表明，经过 PoEt 预训练的模型在数值推理、逻辑推理和多跳推理等任务上均取得了显著提升，证明从程序执行器习得的推理能力可以有效迁移到自然语言场景，为"用符号系统增强神经模型推理能力"这一长期目标提供了一条简洁可复现的新路径。
原文 arXiv:2201.11473;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2201.11473v2