Does Gender Matter? Towards Fairness in Dialogue Systems
Haochen Liu1, Jamell Dacon1, Wenqi Fan2, Hui Liu1, Zitao Liu3, Jiliang Tang1 1 Michigan State University, East Lansing, MI, USA 2 The Hong Kong Polytechnic University, Hong Kong 3 TAL Education Group, Beijing, China The corresponding author: Zitao Liu.
Abstract
Recently there are increasing concerns about the fairness of Artificial Intelligence (AI) in real-world applications such as computer vision and recommendations. For example, recognition algorithms in computer vision are unfair to black people such as poorly detecting their faces and inappropriately identifying them as “gorillas”. As one crucial application of AI, dialogue systems have been extensively applied in our society. They are usually built with real human conversational data; thus they could inherit some fairness issues which are held in the real world. However, the fairness of dialogue systems has not been well investigated. In this paper, we perform a pioneering study about the fairness issues in dialogue systems. In particular, we construct a benchmark dataset and propose quantitative measures to understand fairness in dialogue models. Our studies demonstrate that popular dialogue models show significant prejudice towards different genders and races. Besides, to mitigate the bias in dialogue systems, we propose two simple but effective debiasing methods. Experiments show that our methods can reduce the bias in dialogue systems significantly. The dataset and the implemen
中文速览
对话系统（Dialogue System）在日常生活中越来越普及，但它们是否会对不同性别或种族的用户表现出歧视性行为，此前几乎没有人系统研究过。作者围绕这一空白，构建了一个专门用于公平性评估的基准数据集——通过将对话语境中的男性词替换为女性词、将标准英语替换为非裔美国人英语，生成大量"平行语境对"，再借助情感、礼貌性、多样性和刻板印象词汇四项指标，对主流的 Seq2Seq 生成模型和 Transformer 检索模型进行统计检验。实验结果表明，两类模型都存在显著的性别和种族偏见：例如把"他"换成"她"后，回复情感明显变差；把标准英语换成非裔英语后，回复更容易带有攻击性。为此，作者提出了两种简单有效的去偏方法——在训练数据中进行数据增强和在解码阶段加入公平性约束——实验证明这两种方法能大幅降低模型的偏见程度。这项工作填补了对话系统公平性研究的空白，所发布的数据集和代码也为后续研究提供了重要基础。
原文 arXiv:1910.10486;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1910.10486v3