Revealing Persona Biases in Dialogue Systems
Emily Sheng1 Josh Arnold2∗ Zhou Yu3 Kai-Wei Chang4 Nanyun Peng1,4 1 Information Sciences Institute, University of Southern California 2 Computer Science Department, University of California, Davis 3 Computer Science Department, Columbia University 4 Computer Science Department, University of California, Los Angeles Equal contribution
Abstract
Dialogue systems in the form of chatbots and personal assistants are being increasingly integrated into people’s lives. Modern dialogue systems may consider adopting anthropomorphic personas, mimicking societal demographic groups to appear more approachable and trustworthy to users. However, the adoption of a persona can result in the adoption of biases. In this paper, we present the first large-scale study on persona biases in dialogue systems and conduct analyses on personas of different social classes, sexual orientations, races, and genders. We define persona biases as harmful differences in responses (e.g., varying levels of offensiveness, agreement with harmful statements) generated from adopting different demographic personas. Furthermore, we introduce an open-source framework, UnitPersonaBias, to explore and aggregate persona biases in dialogue systems. By analyzing the Blender and DialoGPT dialogue systems, we observe that adopting personas can actually decrease harmful responses, compared to not using any personas. Additionally, we find that persona choices can affect the degree of harms in generated responses and thus should be systematically evaluated before deployment.
中文速览
聊天机器人和个人助手在给自己设定"人物角色"（persona）时，比如声称自己是某个特定性别、种族或阶层的人，这种设定是否会让它生成带有偏见甚至有害的回复？研究者首次在大规模层面系统研究了这一问题，覆盖性别、种族、性取向和社会阶层等多个维度，并开发了一套名为 UnitPersonaBias 的开源测试框架，通过冒犯性语言、有害附和、职业刻板印象和性别代词等四类指标来量化"角色偏见"。对 Blender 和 DialoGPT 两个主流对话模型的分析发现：给模型赋予一个明确角色反而能减少有害输出，但不同角色在有害程度上的差异依然显著，对特定人群的伤害程度也随角色设定的变化而变化。这一发现表明，对话系统在上线前必须对其角色设定进行系统性偏见评估，否则可能在数百万用户中无声地强化社会偏见和刻板印象。
原文 arXiv:2104.08728;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2104.08728v2