Document Contradiction Detection on ContraDoc (test)
68.1AccuracyDirect Prompting (DP)
Evaluation results
| Method | Links | |||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Direct Prompting (DP)Backbone=LLaMA3.2-90B, Temperature=02026.01 | 68.1 | — | — | 61.6 | — | — | — | — | — | — | — | — | 4.7 | — | — | — | — | — | — | |
| Redact-and-Retry (RnR)Backbone=LLaMA3.2-90B, Temperature=02026.01 | 68.1 | — | — | 61.6 | — | — | — | — | — | — | — | — | 11.8 | — | — | — | — | — | 2.5 | |
| Redact-and-Retry + Constrained Filter (RnR+CF)Backbone=LLaMA3.2-90B, Filter Type=Constrained, Temperature=02026.01 | 68.1 | — | — | 61.6 | — | — | — | — | — | — | — | — | 4.6 | — | — | — | — | — | — | |
| Direct Prompting (DP)Backbone=GPT-4o, Temperature=02026.01 | 68 | — | — | 76.3 | — | — | — | — | — | — | — | — | 5.1 | — | — | — | — | — | — | |
| Redact-and-Retry (RnR)Backbone=GPT-4o, Temperature=02026.01 | 68 | — | — | 76.3 | — | — | — | — | — | — | — | — | 13.3 | — | — | — | — | — | 2.92 | |
| Redact-and-Retry + Constrained Filter (RnR+CF)Backbone=GPT-4o, Filter Type=Constrained, Temperature=02026.01 | 68 | — | — | 76.3 | — | — | — | — | — | — | — | — | 5.4 | — | — | — | — | — | — | |
| GPT-4oApproach=RnR+UF2026.01 | 67.7 | 65.1 | 68 | 71.2 | 71.2 | 35.6 | 64.4 | 28.8 | 54.8 | 77 | 26.1 | 36.7 | 5.9 | 2.85 | 2.9 | 2.87 | 3.95 | 3.88 | 3.92 | |
| Redact-and-Retry + Unconstrained Filter (RnR+UF)Backbone=GPT-4o, Filter Type=Unconstrained, Temperature=02026.01 | 67.7 | — | — | 71.2 | — | — | — | — | — | — | — | — | 5.9 | — | — | — | — | — | — | |
| LLaMA3.2-90BApproach=RnR+UF2026.01 | 67.5 | 70.9 | 64 | 58.3 | 58.3 | 23.5 | 76.5 | 41.7 | 40.2 | 69 | 19.5 | 33.6 | 5 | 2.71 | 2.84 | 2.75 | 3.47 | 3.55 | 3.5 | |
| Redact-and-Retry + Unconstrained Filter (RnR+UF)Backbone=LLaMA3.2-90B, Filter Type=Unconstrained, Temperature=02026.01 | 67.5 | — | — | 58.3 | — | — | — | — | — | — | — | — | 5 | — | — | — | — | — | — | |
| LLaMA3-70BApproach=RnR+UF2026.01 | 60.1 | 86.9 | 35.7 | 22.5 | 22.5 | 3.29 | 96.7 | 77.5 | 19.8 | 88.2 | 10.1 | 45.2 | 1.6 | 2.14 | 3.14 | 2.27 | 2.3 | 3.14 | 2.41 | |
| Direct Prompting (DP)Backbone=LLaMA3-70B, Temperature=02026.01 | 60.1 | — | — | 22.5 | — | — | — | — | — | — | — | — | 1.6 | — | — | — | — | — | — | |
| Redact-and-Retry (RnR)Backbone=LLaMA3-70B, Temperature=02026.01 | 60.1 | — | — | 22.5 | — | — | — | — | — | — | — | — | 2.2 | — | — | — | — | — | 1.41 | |
| Redact-and-Retry + Unconstrained Filter (RnR+UF)Backbone=LLaMA3-70B, Filter Type=Unconstrained, Temperature=02026.01 | 60.1 | — | — | 22.5 | — | — | — | — | — | — | — | — | 1.6 | — | — | — | — | — | — | |
| Redact-and-Retry + Constrained Filter (RnR+CF)Backbone=LLaMA3-70B, Filter Type=Constrained, Temperature=02026.01 | 60.1 | — | — | 22.5 | — | — | — | — | — | — | — | — | 1.5 | — | — | — | — | — | — |