Representing Numbers in NLP: a Survey and a Vision
Avijit Thawani Jay Pujara Pedro Szekely Filip Ilievski University of Southern California Information Sciences Institute
Abstract
NLP systems rarely give special consideration to numbers found in text. This starkly contrasts with the consensus in neuroscience that, in the brain, numbers are represented differently from words. We arrange recent NLP work on numeracy into a comprehensive taxonomy of tasks and methods. We break down the subjective notion of numeracy into 7 subtasks, arranged along two dimensions: granularity (exact vs approximate) and units (abstract vs grounded). We analyze the myriad representational choices made by 18 previously published number encoders and decoders. We synthesize best practices for representing numbers in text and articulate a vision for holistic numeracy in NLP, comprised of design trade-offs and a unified evaluation.
中文速览
数字在自然语言中无处不在，但主流NLP系统长期以来把数字当普通词汇对待——要么直接过滤掉，要么用子词切分成毫无意义的碎片，导致模型在涉及数值推理的任务上表现远差于文本跨度类任务。这篇综述把"数值理解能力"（numeracy）拆解为7个子任务，并从粒度（精确vs近似）和单位（抽象vs有语境）两个维度构建分类体系，同时系统梳理了18种已有数字编解码方案，按字符串表示与实数值表示两大路线归纳其设计差异。通过横向对比各方法在不同任务上的表现，作者提炼出若干最佳实践——例如按位tokenization、科学计数法表示、对数尺度建模等策略普遍优于默认的子词方案——并指出当前研究碎片化、缺乏统一评测基准的问题。这项工作填补了NLP领域对数值表示系统性认识的空白，为后续构建真正"懂数字"的语言模型提供了清晰的路线图。
原文 arXiv:2103.13136;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2103.13136v1