Putting Humans in the Natural Language Processing Loop: A Survey
Zijie J. Wang Dongjin Choi Shenyu Xu Diyi Yang College of Computing, Georgia Institute of Technology {jayw, jin.choi, shenyuxu, denotes equal contribution
Abstract
How can we design Natural Language Processing (NLP) systems that learn from human feedback? There is a growing research body of Human-in-the-loop (HITL) NLP frameworks that continuously integrate human feedback to improve the model itself. HITL NLP research is nascent but multifarious—solving various NLP problems, collecting diverse feedback from different people, and applying different methods to learn from collected feedback. We present a survey of HITL NLP work from both Machine Learning (ML) and Human-Computer Interaction (HCI) communities that highlights its short yet inspiring history, and thoroughly summarize recent frameworks focusing on their tasks, goals, human interactions, and feedback learning methods. Finally, we discuss future directions for integrating human feedback in the NLP development loop.
中文速览
用人类反馈来持续改进NLP模型，是当前人工智能领域的重要挑战——传统的NLP开发流程是线性的，一旦部署就很难吸收真实用户的意见。这篇综述系统梳理了"人在环路"（Human-in-the-Loop, HITL）NLP这一新兴研究方向，汇总了来自机器学习和人机交互两个社区的大量工作，覆盖文本分类、解析、主题建模、摘要生成、机器翻译、对话与问答等多类任务。作者从任务类型、系统目标、人机交互方式（图形界面或自然语言界面）以及反馈学习方法（主动学习、强化学习等）四个维度对现有框架进行了全面归纳，发现引入人类反馈不仅能显著提升模型准确率和鲁棒性，还能增强可解释性并赢得用户信任。这是该领域首篇综合性综述，为希望进入这一方向的研究者提供了清晰的知识地图，并指出了未来值得深耕的研究空白。
原文 arXiv:2103.04044;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2103.04044v1