# Span-ConveRT: Few-shot Span Extraction for Dialog with Pretrained Conversational Representations

Source: [Span-ConveRT: Few-shot Span Extraction for Dialog with Pretrained Conversational Representations](<https://www.sota2.com/research/paper/68fa67f61e209131cea196e7>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T22:25:59.816Z

Authors: Sam Coope, Tyler Farghly, Daniela Gerz, Ivan Vuli\\'c, Matthew Henderson

Published: 2020-05-18T16:40:30.000Z

## Abstract

We introduce Span-ConveRT, a light-weight model for dialog slot-filling which frames the task as a turn-based span extraction task. This formulation allows for a simple integration of conversational knowledge coded in large pretrained conversational models such as ConveRT (Henderson et al., 2019). We show that leveraging such knowledge in Span-ConveRT is especially useful for few-shot learning scenarios: we report consistent gains over 1) a span extractor that trains representations from scratch in the target domain, and 2) a BERT-based span extractor. In order to inspire more work on span extraction for the slot-filling task, we also release RESTAURANTS-8K, a new challenging data set of 8,198 utterances, compiled from actual conversations in the restaurant booking domain.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Slot Filling on Restaurant-8K](<https://www.sota2.com/research/sota/slot-filling-on-restaurant-8k.md>): Slot Filling; Restaurant-8K; F1 Score: 96.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2005.08866>)

[PDF](<https://arxiv.org/pdf/2005.08866.pdf>)