# Contradiction Detection on RevCI (test)

Source: [Contradiction Detection on RevCI (test)](<https://www.sota2.com/research/sota/contradiction-detection-on-revci-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-12T14:52:40.847Z

Evaluation of Contradiction Detection performance on the test split of the RevCI dataset, measured using False Negative Rate (FNR) and False Positive Rate (FPR).

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Contradiction Detection |
| Dataset (including any stated split) | RevCI (test) |
| Metric | FNR |
| Unit | PERCENT |
| Direction | Lower is better |
| Results loaded | 14 of 14 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [FNR](<https://www.sota2.com/research/sota/contradiction-detection-on-revci-test.md?metric=FNR>): Lower is better; unit: PERCENT.

- [FPR](<https://www.sota2.com/research/sota/contradiction-detection-on-revci-test.md?metric=FPR>): Lower is better; unit: PERCENT.

## Recorded results

| Method | FNR | Result date | Report paper |
| --- | --- | --- | --- |
| IMPACT-P | 19.01 | Not recorded | 6a02b49adbc836b87d824df8 |
| IMPACT-OA | 23.9 | Not recorded | 6a02b49adbc836b87d824df8 |
| CourtEval | 25.2 | Not recorded | 6a02b49adbc836b87d824df8 |
| ChatEval | 25.8 | Not recorded | 6a02b49adbc836b87d824df8 |
| LLaMA-4 Maverick | 26.39 | Not recorded | 6a02b49adbc836b87d824df8 |
| MAD | 26.9 | Not recorded | 6a02b49adbc836b87d824df8 |
| Self-Refine | 27.5 | Not recorded | 6a02b49adbc836b87d824df8 |
| Debate | 28.7 | Not recorded | 6a02b49adbc836b87d824df8 |
| GPT-5.2 | 29.35 | Not recorded | 6a02b49adbc836b87d824df8 |
| LLaMA-3-8B-Instruct | 35.7 | Not recorded | 6a02b49adbc836b87d824df8 |
| Gemini-3 Flash | 37.41 | Not recorded | 6a02b49adbc836b87d824df8 |
| TIDE | 37.71 | Not recorded | 6a02b49adbc836b87d824df8 |
| ContraSciView | 39.2 | Not recorded | 6a02b49adbc836b87d824df8 |
| Qwen-2.5-7B-Instruct | 41.27 | Not recorded | 6a02b49adbc836b87d824df8 |

### IMPACT-P

FNR: 19.01. Result ID: 6a033e3c47f34ce924e82e3e.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b49adbc836b87d824df8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a02b49adbc836b87d824df8.md>)

Recorded evaluation configuration: {"Category":"Our Proposed"}. This record does not assert independent reproduction.

### IMPACT-OA

FNR: 23.9. Result ID: 6a033e3c47f34ce924e82e3d.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b49adbc836b87d824df8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a02b49adbc836b87d824df8.md>)

Recorded evaluation configuration: {"Category":"Our Proposed"}. This record does not assert independent reproduction.

### CourtEval

FNR: 25.2. Result ID: 6a033e3c47f34ce924e82e3c.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b49adbc836b87d824df8.md>)

Recorded evaluation configuration: {"Category":"Generic multi-agent"}. This record does not assert independent reproduction.

### ChatEval

FNR: 25.8. Result ID: 6a033e3c47f34ce924e82e3b.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b49adbc836b87d824df8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa705a1e209131cea4a819.md>)

Recorded evaluation configuration: {"Category":"Generic multi-agent"}. This record does not assert independent reproduction.

### LLaMA-4 Maverick

FNR: 26.39. Result ID: 6a033e3c47f34ce924e82e33.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b49adbc836b87d824df8.md>)

Recorded evaluation configuration: {"Category":"Single-agent (CoT)"}. This record does not assert independent reproduction.

### MAD

FNR: 26.9. Result ID: 6a033e3c47f34ce924e82e3a.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b49adbc836b87d824df8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75171e209131cea65abb.md>)

Recorded evaluation configuration: {"Category":"Generic multi-agent"}. This record does not assert independent reproduction.

### Self-Refine

FNR: 27.5. Result ID: 6a033e3c47f34ce924e82e38.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b49adbc836b87d824df8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f7f1e209131cea45fc4.md>)

Recorded evaluation configuration: {"Category":"Generic multi-agent"}. This record does not assert independent reproduction.

### Debate

FNR: 28.7. Result ID: 6a033e3c47f34ce924e82e39.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b49adbc836b87d824df8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73501e209131cea5bc34.md>)

Recorded evaluation configuration: {"Category":"Generic multi-agent"}. This record does not assert independent reproduction.

### GPT-5.2

FNR: 29.35. Result ID: 6a033e3c47f34ce924e82e32.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b49adbc836b87d824df8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa727a1e209131cea56a1e.md>)

[Implementation](<https://github.com/openai/evals>)

Recorded evaluation configuration: {"Category":"Single-agent (CoT)"}. This record does not assert independent reproduction.

### LLaMA-3-8B-Instruct

FNR: 35.7. Result ID: 6a033e3c47f34ce924e82e35.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b49adbc836b87d824df8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75d11e209131cea6a028.md>)

[Implementation](<https://github.com/meta-llama/llama-agentic-system>)

Recorded evaluation configuration: {"Category":"Single-agent (CoT)"}. This record does not assert independent reproduction.

### Gemini-3 Flash

FNR: 37.41. Result ID: 6a033e3c47f34ce924e82e34.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b49adbc836b87d824df8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78731e209131cea78622.md>)

Recorded evaluation configuration: {"Category":"Single-agent (CoT)"}. This record does not assert independent reproduction.

### TIDE

FNR: 37.71. Result ID: 6a033e3c47f34ce924e82e3f.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b49adbc836b87d824df8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a02b49adbc836b87d824df8.md>)

Recorded evaluation configuration: {"Category":"Our Proposed"}. This record does not assert independent reproduction.

### ContraSciView

FNR: 39.2. Result ID: 6a033e3c47f34ce924e82e37.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b49adbc836b87d824df8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa71041e209131cea4e96a.md>)

Recorded evaluation configuration: {"Category":"Single-agent (CoT)"}. This record does not assert independent reproduction.

### Qwen-2.5-7B-Instruct

FNR: 41.27. Result ID: 6a033e3c47f34ce924e82e36.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b49adbc836b87d824df8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Category":"Single-agent (CoT)"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
