Generate & Rank: A Multi-task Framework for Math Word Problems
Jianhao Shen1†, Yichun Yin2, Lin Li3, Lifeng Shang2, Xin Jiang2, Ming Zhang1*, Qun Liu2 1Department of Computer Science, School of EECS, Peking University 2Huawei Noah’s Ark Lab 3Huawei HiSilicon {jhshen, {yinyichun, lilin29, shang.lifeng, jiang.xin, † This work is done when Jianhao Shen is an intern at Huawei Noah’s Ark Lab*Corresponding author
Abstract
Math word problem (MWP) is a challenging and critical task in natural language processing. Many recent studies formalize MWP as a generation task and have adopted sequence-to-sequence models to transform problem descriptions to mathematical expressions. However, mathematical expressions are prone to minor mistakes while the generation objective does not explicitly handle such mistakes. To address this limitation, we devise a new ranking task for MWP and propose Generate & Rank, a multi-task framework based on a generative pre-trained language model. By joint training with generation and ranking, the model learns from its own mistakes and is able to distinguish between correct and incorrect expressions. Meanwhile, we perform tree-based disturbance specially designed for MWP and an online update to boost the ranker. We demonstrate the effectiveness of our proposed method on the benchmark and the results show that our method consistently outperforms baselines in all datasets. Particularly, in the classical Math23k, our method is 7% (78.4% $\rightarrow$ 85.4%) higher than the state-of-the-art111Code will be available soon..
中文速览
数学应用题（Math Word Problem）求解的难点在于，表达式哪怕只差一个符号或数字，答案就会完全错误，而传统的序列生成模型只靠最大化正确答案的生成概率来训练，根本没有让模型学会"辨别对错"的能力。为此，研究者提出了"生成+排序"（Generate & Rank）多任务框架：先用基于BART的生成器产出多个候选表达式，再训练一个排序器从中挑出最可能正确的那个，两者共享参数并联合优化。为了给排序器提供高质量的训练样本，他们还专门设计了针对表达式语法树的四种扰动操作（扩展、编辑、删除、交换），并在每轮训练后在线更新候选池，让模型不断从自己的错误中学习。在经典中文数学题数据集Math23K上，该方法将准确率从78.4%提升至85.4%，超出此前最优方法7个百分点，尤其在长表达式生成上改善显著，证明"生成+排序"的思路是解决数学应用题的一条更可靠的路径。
原文 arXiv:2109.03034;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2109.03034v1