What’s in the Box? A Preliminary Analysis of Undesirable Content in the Common Crawl Corpus
Alexandra (Sasha) Luccioni Université de Montréal、Mila Québec AI Institute \AndJoseph D. Viviano Mila Québec AI Institute
Abstract
Whereas much of the success of the current generation of neural language models has been driven by increasingly large training corpora, relatively little research has been dedicated to analyzing these massive sources of textual data. In this exploratory analysis, we delve deeper into the Common Crawl, a colossal web corpus that is extensively used for training language models. We find that it contains a significant amount of undesirable content, including hate speech and sexually explicit content, even after filtering procedures. We discuss the potential impacts of this content on language models and conclude with future research directions and a more mindful approach to corpus collection and analysis.
中文速览
大规模语言模型的训练高度依赖Common Crawl这一每月抓取数百TB网页文本的超大规模语料库，但这个"数据基础"里究竟装着什么，学界却鲜有深究。研究者随机抽取了Common Crawl约1%的文件（约1.15亿字节、逾580万网页），用仇恨言论检测、色情内容识别和基于困惑度（perplexity）的过滤三种手段对其进行分析，结果发现：即便经过常规过滤，仍有约4%–6%的网页含有仇恨言论，约2%含有色情内容；而常用的困惑度过滤虽能部分去除色情内容，却对仇恨言论几乎无效。这意味着GPT、BERT等主流大模型所依赖的训练数据中，存在不可忽视的有害内容残留，一旦被模型"学进去"，便可能在下游应用中被触发复现，对真实用户造成伤害。这项工作提醒整个NLP社区：在追求数据规模的同时，必须把语料质量与内容审查提升到同等重要的位置，并呼吁开发更系统、可扩展的有害内容检测与过滤方法。
原文 arXiv:2105.02732;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2105.02732v3