Mapping Natural-language Problems to Formal-language Solutions Using Structured Neural Representations
Kezhen Chen Qiuyuan Huang Hamid Palangi Paul Smolensky Kenneth D. Forbus Jianfeng Gao
Abstract
Generating formal-language programs represented by relational tuples, such as Lisp programs or mathematical operations, to solve problems stated in natural language is a challenging task because it requires explicitly capturing discrete symbolic structural information implicit in the input. However, most general neural sequence models do not explicitly capture such structural information, limiting their performance on these tasks. In this paper, we propose a new encoder-decoder model based on a structured neural representation, Tensor Product Representations (TPRs), for mapping Natural-language problems to Formal-language solutions, called TP-N2F. The encoder of TP-N2F employs TPR ‘binding’ to encode natural-language symbolic structure in vector space and the decoder uses TPR ‘unbinding’ to generate, in symbolic space, a sequential program represented by relational tuples, each consisting of a relation (or operation) and a number of arguments. TP-N2F considerably outperforms LSTM-based seq2seq models on two benchmarks and creates new state-of-the-art results. Ablation studies show that improvements can be attributed to the use of structured TPRs explicitly in both the encoder and d
中文速览
研究者希望让神经网络能把自然语言描述的数学或编程题，自动转化为由关系元组（如操作符加参数）组成的形式化程序，难点在于模型需要显式捕捉语言和程序中隐含的离散符号结构。他们提出了一种基于张量积表示（Tensor Product Representation，TPR）的编码-解码模型TP-N2F：编码器用TPR"绑定"操作把自然语言的角色-填充结构嵌入向量空间，解码器再用TPR"解绑定"操作从向量中逐步还原出关系元组形式的程序。在MathQA等两个基准测试上，TP-N2F显著超越了基于LSTM的序列到序列模型并刷新了最优成绩，消融实验也证明性能提升直接来自编解码两端对TPR结构的明确使用。这项工作的意义在于，它首次将TPR绑定与解绑定结合进一个端到端的生成框架，为需要符号推理的自然语言理解任务提供了兼具性能与可解释性的新思路。
原文 arXiv:1910.02339;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1910.02339v3