Social Biases in NLP Models as Barriers for Persons with Disabilities
Ben Hutchinson, Vinodkumar Prabhakaran, Emily Denton, Kellie Webster, Yu Zhong, Stephen Denuyl Google
Abstract
Building equitable and inclusive NLP technologies demands consideration of whether and how social attitudes are represented in ML models. In particular, representations encoded in models often inadvertently perpetuate undesirable social biases from the data on which they are trained. In this paper, we present evidence of such undesirable biases towards mentions of disability in two different English language models: toxicity prediction and sentiment analysis. Next, we demonstrate that the neural embeddings that are the critical first step in most NLP pipelines similarly contain undesirable biases towards mentions of disability. We end by highlighting topical biases in the discourse about disability which may contribute to the observed model biases; for instance, gun violence, homelessness, and drug addiction are over-represented in texts discussing mental illness.
中文速览
公平且无障碍的 NLP 技术需要认真审视模型是否在无意间放大了社会偏见——针对残障群体的偏见尤其被长期忽视。研究者收集了56种英语中指代残障人士的表达方式，通过"扰动测试"（perturbation test）将这些表达插入真实句子，发现毒性检测和情感分析两个主流模型在遇到残障相关词汇时会系统性地给出更高毒性评分或更负面情感评分，且使用"非推荐"称谓（如带有贬义的俚语）时偏差更为明显。在词向量层面，广泛使用的预训练模型 BERT 在做完形填空时，针对残障相关短语预测出的词汇也呈现出更高比例的负面情感。通过分析一个大规模真实语料库，研究者发现，精神疾病话题在公共讨论中频繁与无家可归、枪支暴力、毒品成瘾等负面议题共现，这很可能正是上述模型偏见的根源所在。这项工作揭示了一条从社会偏见到训练数据、再到模型输出的完整传导链条，提醒业界在将 NLP 系统用于内容审核、情感分析等场景时，必须警惕其对残障群体话语权和社会可见度的潜在伤害。
原文 arXiv:2005.00813;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2005.00813v1