MWPToolkit: An Open-source Framework for Deep Learning-based Math Word Problem Solvers
Yihuai Lan1, Lei Wang, Qiyuan Zhang, Yunshi Lan3, Bing Tian Dai2, Yan Wang4, Dongxiang Zhang5, Ee-Peng Lim2 1 Xihua University, 2Singapore Management University 3East China Normal University, 4 Tencent AI Lab, 5 Zhejiang University {lei.wang.2019, {btdai, {yifan2250, Equal contribution. Corresponding author.
Abstract
Developing automatic Math Word Problem (MWP) solvers has been an interest of NLP researchers since the 1960s. Over the last few years, there are a growing number of datasets and deep learning-based methods proposed for effectively solving MWPs. However, most existing methods are benchmarked solely on one or two datasets, varying in different configurations, which leads to a lack of unified, standardized, fair, and comprehensive comparison between methods. This paper presents MWPToolkit, the first open-source framework for solving MWPs. In MWPToolkit, we decompose the procedure of existing MWP solvers into multiple core components and decouple their models into highly reusable modules. We also provide a hyper-parameter search function to boost the performance. In total, we implement and compare 17 MWP solvers on 4 widely-used single equation generation benchmarks and 2 multiple equations generation benchmarks. These features enable our MWPToolkit to be suitable for researchers to reproduce advanced baseline models and develop new MWP solvers quickly. Code and documents are available at https://github.com/LYH-YF/MWPToolkit.
中文速览
数学应用题（Math Word Problem, MWP）自动求解是自然语言处理领域几十年来的经典难题，但现有深度学习方法各自为战，在不同数据集和评测设置下单独汇报结果，导致方法之间根本无法公平比较。为此，研究者构建了 MWPToolkit——首个面向 MWP 求解的开源统一框架，将数据预处理、模型、评测等核心环节解耦为可复用模块，并内置超参数搜索功能，让使用者只需简单配置文件即可运行或扩展模型。框架在 6 个主流数据集上对 17 种求解模型（涵盖 Seq2Seq、Seq2Tree、Graph2Tree 及预训练语言模型等类别）进行了统一实验，结果显示 Graph2Tree 和 RoBERTaGen 是目前最强的基线方法。MWPToolkit 的意义在于为领域研究提供了一个标准化的公平比较平台，大幅降低了复现已有方法、开发新模型的门槛，有望推动数学应用题自动求解研究更快向前发展。
原文 arXiv:2109.00799;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2109.00799v2