Language selection

Government of Canada / Gouvernement du Canada

Search

On cross-dataset generalization in automatic detection of online abuse

From National Research Council Canada

Download
  1. (PDF, 679 KiB)
  2. (PDF, 641 KiB)
AuthorSearch for: 1; Search for: 1
Affiliation
  1. National Research Council of Canada. Digital Technologies
FormatText, Article
ConferenceWOAH 2020 - Fourth Workshop on Online Abuse and Harms, November 20th, 2020 - [Held Online]
Abstract
NLP research has attained high performances in abusive language detection as a supervised classification task. While in research settings, training and test datasets are usually obtained from similar data samples, in practice systems are often applied on data that are different from the training set in topic and class distributions. Also, the ambiguity in class definitions inherited in this task aggravates the discrepancies between source and target datasets. We explore the topic bias and the task formulation bias in cross-dataset generalization. We show that the benign examples in the Wikipedia Detox dataset are biased towards platformspecific topics. We identify these examples using unsupervised topic modeling and manual inspection of topics’ keywords. Removing these topics increases cross-dataset generalization, without reducing in-domain classification performance. For a robust dataset design, we suggest applying inexpensive unsupervised methods to inspect the collected data and downsize the non-generalizable content before manually annotating for class labels.
Publication date
Date created2020-11-30
PublisherAssociation for Computational Linguistics
Licence
In
LanguageEnglish
Peer reviewedYes
Export citationExport as RIS
Report a correctionReport a correction (opens in a new tab)
Record identifier846aa815-b9d6-4b34-9501-9163df950d7a
Record created2020-11-30
Record modified2020-12-02
Date modified: