SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)
Marcos Zampieri,1 Shervin Malmasi,2 Preslav Nakov,3 Sara Rosenthal,4 Noura Farra,5 Ritesh Kumar6 1University of Wolverhampton, UK, 2Amazon Research, USA 3Qatar Computing Research Institute, HBKU, Qatar 4IBM Research, USA, 5Columbia University, USA, 6Bhim Rao Ambedkar University, India
Abstract
We present the results and the main findings of SemEval-2019 Task 6 on Identifying and Categorizing Offensive Language in Social Media (OffensEval). The task was based on a new dataset, the Offensive Language Identification Dataset (OLID), which contains over 14,000 English tweets. It featured three sub-tasks. In sub-task A, the goal was to discriminate between offensive and non-offensive posts. In sub-task B, the focus was on the type of offensive content in the post. Finally, in sub-task C, systems had to detect the target of the offensive posts. OffensEval attracted a large number of participants and it was one of the most popular tasks in SemEval-2019. In total, about 800 teams signed up to participate in the task, and 115 of them submitted results, which we present and analyze in this report.
中文速览
社交媒体上的攻击性语言泛滥，人工审核既耗时又会对审核者造成心理伤害，因此自动识别这类内容至关重要。SemEval-2019第6号任务（OffensEval）围绕专门构建的攻击性语言识别数据集（OLID）展开，该数据集包含超过14,000条英文推文，按三个层级进行标注：是否具有攻击性、攻击类型（有无具体目标），以及攻击对象（个人、群体还是其他）。约800支队伍报名参赛，最终115支提交了结果，参赛系统涵盖传统机器学习（如SVM）到深度学习（如CNN、BiLSTM），其中基于BERT的模型在多个子任务中表现最为突出，最优系统在子任务A中取得了82.9%的宏平均F1分数。这项任务不仅是SemEval-2019最受关注的比赛之一，其分层标注框架也首次将攻击类型与攻击目标结合考量，为仇恨言论、网络欺凌等细分问题的统一研究提供了重要资源。
原文 arXiv:1903.08983;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1903.08983v3