# A Large-Scale Corpus for Conversation Disentanglement

Source: [A Large-Scale Corpus for Conversation Disentanglement](<https://www.sota2.com/research/paper/68fa67a71e209131cea177d9>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T02:59:50.304Z

Authors: Jonathan K. Kummerfeld, Sai R. Gouravajhala, Joseph Peper, Vignesh Athreya, Chulaka Gunasekara, Jatin Ganhotra, Siva Sankalp Patel, Lazaros Polymenakos, Walter S. Lasecki

Published: 2018-10-25T21:44:14.000Z

## Abstract

Disentangling conversations mixed together in a single stream of messages is a difficult task, made harder by the lack of large manually annotated datasets. We created a new dataset of 77,563 messages manually annotated with reply-structure graphs that both disentangle conversations and define internal conversation structure. Our dataset is 16 times larger than all previously released datasets combined, the first to include adjudication of annotation disagreements, and the first to include context. We use our data to re-examine prior work, in particular, finding that 80% of conversations in a widely used dialogue corpus are either missing messages or contain extra messages. Our manually-annotated data presents an opportunity to develop robust data-driven methods for conversation disentanglement, which will help advance dialogue research.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Conversation Disentanglement on Ubuntu (test)](<https://www.sota2.com/research/sota/conversation-disentanglement-on-ubuntu-test.md>): Conversation Disentanglement; Ubuntu (test); VI: 91.5.

- [Conversation Disentanglement on Channel Two Elsner (test)](<https://www.sota2.com/research/sota/conversation-disentanglement-on-channel-two-elsner-test.md>): Conversation Disentanglement; Channel Two Elsner (test); 1-1 Score: 57.5.

- [Conversation Disentanglement on Channel Two Ubuntu/Ours (test)](<https://www.sota2.com/research/sota/conversation-disentanglement-on-channel-two-ubuntu-ours-test.md>): Conversation Disentanglement; Channel Two Ubuntu/Ours (test); 1-1 Score: 62.8.

- [Dialogue Disentanglement on Ubuntu IRC (test)](<https://www.sota2.com/research/sota/dialogue-disentanglement-on-ubuntu-irc-test.md>): Dialogue Disentanglement; Ubuntu IRC (test); VI: 91.5.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1810.11118>)

[PDF](<https://arxiv.org/pdf/1810.11118.pdf>)

[Implementation](<https://jkk.name/irc-disentanglement>)