# When Reviews Disagree: Fine-Grained Contradiction Analysis in Scientific Peer Reviews

Source: [When Reviews Disagree: Fine-Grained Contradiction Analysis in Scientific Peer Reviews](<https://www.sota2.com/research/paper/6a02b49adbc836b87d824df8>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-13T05:00:20.520Z

Authors: Sandeep Kumar, Yash Kamdar, Abid Hossain, Bharti Kumari, Tanik Saikh, Asif Ekbal

Published: 2026-05-11T08:20:58.000Z

## Abstract

Scientific peer reviews frequently contain conflicting expert judgments, and the increasing scale of conference submissions makes it challenging for Area Chairs and editors to reliably identify and interpret such disagreements. Existing approaches typically frame reviewer disagreement as binary contradiction detection over isolated sentence pairs, abstracting away the review-level context and obscuring differences in the severity of evaluative conflict. In this work, we introduce a fine-grained formulation of reviewer contradiction analysis that operates over full peer reviews by explicitly identifying contradiction evidence spans and assigning graded disagreement intensity scores. To support this task, we present RevCI, an expert-annotated benchmark of peer-review pairs with evidence-level contradiction annotations with graded intensity labels. We further propose IMPACT, a structured multi-agent framework that integrates aspect-conditioned evidence extraction, deliberative reasoning, and adjudication to model reviewer contradictions and their intensity. To support efficient deployment, we distill IMPACT into TIDE, a small language model that predicts contradiction evidence and intensity in a single forward pass. Experimental results show that IMPACT substantially outperforms strong single-agent and generic multi-agent baselines in both evidence identification and intensity agreement, while TIDE achieves competitive performance at significantly lower inference cost.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Contradiction Detection on RevCI (test)](<https://www.sota2.com/research/sota/contradiction-detection-on-revci-test.md>): Contradiction detection; RevCI (test); FNR: 19.01.

- [Contradiction Intensity Agreement on RevCI (test)](<https://www.sota2.com/research/sota/contradiction-intensity-agreement-on-revci-test.md>): Contradiction Intensity Agreement; RevCI (test); Kappa (κ): 0.3862.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2605.10171>)

[PDF](<https://arxiv.org/pdf/2605.10171.pdf>)
