# Abuse is Contextual, What about NLP? The Role of Context in Abusive Language Annotation and Detection

Source: [Abuse is Contextual, What about NLP? The Role of Context in Abusive Language Annotation and Detection](<https://www.sota2.com/research/paper/68fa69a01e209131cea232ff>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T23:04:46.235Z

Authors: Stefano Menini, Alessio Palmero Aprosio, Sara Tonelli

Published: 2021-03-27T14:31:52.000Z

## Abstract

The datasets most widely used for abusive language detection contain lists of messages, usually tweets, that have been manually judged as abusive or not by one or more annotators, with the annotation performed at message level. In this paper, we investigate what happens when the hateful content of a message is judged also based on the context, given that messages are often ambiguous and need to be interpreted in the context of occurrence. We first re-annotate part of a widely used dataset for abusive language detection in English in two conditions, i.e. with and without context. Then, we compare the performance of three classification algorithms obtained on these two types of dataset, arguing that a context-aware classification is more challenging but also more similar to a real application scenario.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Toxicity Detection on CAD full](<https://www.sota2.com/research/sota/toxicity-detection-on-cad-full.md>): Toxicity Detection; CAD (full); F1 Score: 50.9.

- [Toxicity Detection on HQG](<https://www.sota2.com/research/sota/toxicity-detection-on-hqg.md>): Toxicity Detection; HQG; F1 Score: 90.7.

- [Toxicity Detection on HQR](<https://www.sota2.com/research/sota/toxicity-detection-on-hqr.md>): Toxicity Detection; HQR; F1 Score: 79.9.

- [Toxicity Detection on CAD context](<https://www.sota2.com/research/sota/toxicity-detection-on-cad-context.md>): Toxicity Detection; CAD context; F1 Score: 55.3.

- [Toxicity Detection on BBF](<https://www.sota2.com/research/sota/toxicity-detection-on-bbf.md>): Toxicity Detection; BBF; F1 Score: 62.3.

- [Toxicity Detection on BAD](<https://www.sota2.com/research/sota/toxicity-detection-on-bad.md>): Toxicity Detection; BAD; F1 Score: 74.7.

- [Toxicity Detection on FBK full](<https://www.sota2.com/research/sota/toxicity-detection-on-fbk-full.md>): Toxicity Detection; FBK full; F1 Score: 39.

- [Toxicity Detection on FBK flipped](<https://www.sota2.com/research/sota/toxicity-detection-on-fbk-flipped.md>): Toxicity Detection; FBK flipped; F1 Score: 19.2.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2103.14916>)

[PDF](<https://arxiv.org/pdf/2103.14916.pdf>)