# COVID-Twitter-BERT: A Natural Language Processing Model to Analyse COVID-19 Content on Twitter

Source: [COVID-Twitter-BERT: A Natural Language Processing Model to Analyse COVID-19 Content on Twitter](<https://www.sota2.com/research/paper/68fa67881e209131cea16bc2>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T22:16:13.371Z

Authors: Martin M\\"uller, Marcel Salath\\'e, Per E Kummervold

Published: 2020-05-15T12:40:46.000Z

## Abstract

In this work, we release COVID-Twitter-BERT (CT-BERT), a transformer-based model, pretrained on a large corpus of Twitter messages on the topic of COVID-19. Our model shows a 10-30% marginal improvement compared to its base model, BERT-Large, on five different classification datasets. The largest improvements are on the target domain. Pretrained transformer models, such as CT-BERT, are trained on a specific target domain and can be used for a wide variety of natural language processing tasks, including classification, question-answering and chatbots. CT-BERT is optimised to be used on COVID-19 content, in particular social media posts from Twitter.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Informative Tweet Identification on WNUT Task 2 2020 (CV)](<https://www.sota2.com/research/sota/informative-tweet-identification-on-wnut-task-2-2020-cv.md>): Informative Tweet Identification; WNUT Task 2 2020 (CV); F1 Score: 96.22.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2005.07503>)

[PDF](<https://arxiv.org/pdf/2005.07503.pdf>)
