Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection
Bertie Vidgen†, Tristan Thrush‡, Zeerak Waseem⋆, Douwe Kiela‡ †The Alan Turing Institute; ⋆University of Sheffield; ‡Facebook AI Research
Abstract
We present a human-and-model-in-the-loop process for dynamically generating datasets and training better performing and more robust hate detection models. We provide a new dataset of ${\sim}40,000$ entries, generated and labelled by trained annotators over four rounds of dynamic data creation. It includes ${\sim}15,000$ challenging perturbations and each hateful entry has fine-grained labels for the type and target of hate. Hateful entries make up 54% of the dataset, which is substantially higher than comparable datasets. We show that model performance is substantially improved using this approach. Models trained on later rounds of data collection perform better on test sets and are harder for annotators to trick. They also perform better on HateCheck, a suite of functional tests for online hate detection. We provide the code, dataset and annotation guidelines for other researchers to use.111Accepted at ACL 2021.
中文速览
仇恨言论（hate speech）在网络上泛滥，但现有自动检测模型普遍存在鲁棒性差、容易被绕过的问题，根源在于训练数据过于静态、缺乏挑战性。研究团队提出了一套"人机协同动态对抗"流程：先用已有数据训练一个基础模型，再让受过专业训练的标注员反复尝试用各种手段"骗过"模型，把模型判断失误的样本收集起来重新训练，如此循环四轮，最终生成了约4万条数据集，其中54%为仇恨内容，并附有类型和攻击对象等细粒度标注，还包含约1.5万条专门设计的"对比扰动样本"（contrast perturbations）。实验显示，随着轮次推进，模型越来越难被骗：标注员的"欺骗成功率"持续下降，而模型在独立功能测试集HateCheck上的准确率从60%大幅提升至95%。这项工作不仅证明了动态对抗式数据构建能显著提升检测模型的性能与泛化能力，还向社区开放了数据集、代码和标注指南，为构建更可靠的内容审核系统奠定了基础。
原文 arXiv:2012.15761;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2012.15761v2