# Long Context Question Answering via Supervised Contrastive Learning

Source: [Long Context Question Answering via Supervised Contrastive Learning](<https://www.sota2.com/research/paper/68fa6c761e209131cea340db>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T00:10:48.263Z

Authors: Avi Caciularu, Ido Dagan, Jacob Goldberger, Arman Cohan

Published: 2021-12-16T10:40:49.000Z

## Abstract

Long-context question answering (QA) tasks require reasoning over a long document or multiple documents. Addressing these tasks often benefits from identifying a set of evidence spans (e.g., sentences), which provide supporting evidence for answering the question. In this work, we propose a novel method for equipping long-context QA models with an additional sequence-level objective for better identification of the supporting evidence. We achieve this via an additional contrastive supervision signal in finetuning, where the model is encouraged to explicitly discriminate supporting evidence sentences from negative ones by maximizing question-evidence similarity. The proposed additional loss exhibits consistent improvements on three different strong long-context transformer models, across two challenging question answering benchmarks -- HotpotQA and QAsper.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2112.08777>)

[PDF](<https://arxiv.org/pdf/2112.08777.pdf>)