Question Answering on ConflictQA
53.8AccuracyCR-DPO
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| CR-DPOBackbone=Llama-3.1-8B2026.05 | 53.8 | 94.9 | 81.6 | 88.2 | |
| SABERBackbone=Llama-3.1-8B2026.05 | 53.5 | 97 | 85 | 91 | |
| SABERBackbone=Qwen-2.5-7B2026.05 | 49.6 | 99.3 | 90 | 94.6 | |
| Implicit-SCRBackbone=Qwen-2.5-7B2026.05 | 47.8 | 94.5 | 36 | 65.3 | |
| CR-DPOBackbone=Qwen-2.5-7B2026.05 | 47.8 | 98.5 | 61 | 79.8 | |
| Implicit-SCRBackbone=Llama-3.1-8B2026.05 | 47.3 | 89.5 | 36.1 | 62.8 | |
| TPCBackbone=Llama-3.1-8B2026.05 | 47 | 92.7 | 34.7 | 63.7 | |
| SABERBackbone=Llama-3.2-3B2026.05 | 46.9 | 97.9 | 82.9 | 90.4 | |
| Vanilla RAGBackbone=Llama-3.1-8B2026.05 | 46.8 | 100 | 7.5 | 53.7 | |
| Vanilla RAGBackbone=Qwen-2.5-7B2026.05 | 44.5 | 100 | 7 | 53.5 | |
| TPCBackbone=Qwen-2.5-7B2026.05 | 44.3 | 96 | 27 | 61.5 | |
| SABERBackbone=Qwen-2.5-3B2026.05 | 44 | 98 | 75.4 | 86.7 | |
| Implicit-SCRBackbone=Qwen-2.5-3B2026.05 | 43.4 | 92.9 | 19.3 | 56.1 | |
| TACS-LRBackbone=Llama-3.1-8B2026.05 | 42.9 | 90.1 | 12.2 | 51.2 | |
| Vanilla RAGBackbone=Qwen-2.5-3B2026.05 | 42.1 | 100 | 7 | 53.5 | |
| Vanilla RAGBackbone=Llama-3.2-3B2026.05 | 42 | 100 | 5.4 | 52.7 | |
| TACS-LRBackbone=Qwen-2.5-7B2026.05 | 41.9 | 84 | 13 | 48.5 | |
| TPCBackbone=Qwen-2.5-3B2026.05 | 41.6 | 96.7 | 40.4 | 68.5 | |
| TPCBackbone=Llama-3.2-3B2026.05 | 41.4 | 92.4 | 37.8 | 65.1 | |
| Implicit-SCRBackbone=Llama-3.2-3B2026.05 | 41.3 | 88.1 | 20.3 | 54.2 | |
| Explicit-SCRBackbone=Llama-3.1-8B2026.05 | 40.9 | 73.9 | 52.4 | 63.1 | |
| CR-DPOBackbone=Llama-3.2-3B2026.05 | 39.8 | 79.2 | 85.6 | 82.4 | |
| TACS-LRBackbone=Qwen-2.5-3B2026.05 | 39.5 | 90.9 | 3.5 | 47.2 | |
| CR-DPOBackbone=Qwen-2.5-3B2026.05 | 38.6 | 92.2 | 31.6 | 61.9 | |
| Internal-EvalBackbone=Qwen-2.5-7B2026.05 | 38.2 | 72.1 | 63 | 67.6 | |
| Internal-EvalBackbone=Llama-3.2-3B2026.05 | 37.2 | 74.1 | 65 | 69.5 | |
| TACS-LRBackbone=Llama-3.2-3B2026.05 | 36.7 | 82.9 | 6.2 | 44.6 | |
| Explicit-SCRBackbone=Qwen-2.5-7B2026.05 | 34.8 | 70.7 | 30 | 50.3 | |
| Internal-EvalBackbone=Llama-3.1-8B2026.05 | 31.9 | 37.6 | 72.1 | 54.9 | |
| Explicit-SCRBackbone=Qwen-2.5-3B2026.05 | 31.8 | 78.4 | 14 | 46.2 | |
| Context-EvalBackbone=Llama-3.1-8B2026.05 | 31.7 | 36.6 | 79.6 | 58.1 | |
| Context-EvalBackbone=Qwen-2.5-7B2026.05 | 31 | 55.9 | 50 | 53 | |
| Internal-EvalBackbone=Qwen-2.5-3B2026.05 | 30.6 | 64.9 | 57.9 | 61.4 | |
| Context-EvalBackbone=Llama-3.2-3B2026.05 | 29.5 | 51.9 | 68.1 | 60 | |
| Explicit-SCRBackbone=Llama-3.2-3B2026.05 | 27.6 | 54.6 | 68.5 | 61.6 | |
| Closed-BookBackbone=Llama-3.1-8B2026.05 | 22.7 | 3 | 100 | 51.5 | |
| R-TuningBackbone=Qwen-2.5-7B2026.05 | 15.9 | 4.4 | 70.9 | 37.7 | |
| Closed-BookBackbone=Qwen-2.5-7B2026.05 | 14.6 | 1.3 | 100 | 50.6 | |
| Closed-BookBackbone=Llama-3.2-3B2026.05 | 13.9 | 0.8 | 100 | 50.4 | |
| Context-EvalBackbone=Qwen-2.5-3B2026.05 | 12.3 | 13.8 | 70.2 | 42 | |
| R-TuningBackbone=Llama-3.1-8B2026.05 | 10.5 | 2.6 | 28.3 | 15.5 | |
| R-TuningBackbone=Qwen-2.5-3B2026.05 | 10.4 | 6.4 | 46.2 | 26.3 | |
| Closed-BookBackbone=Qwen-2.5-3B2026.05 | 9.1 | 1.1 | 100 | 50.6 | |
| R-TuningBackbone=Llama-3.2-3B2026.05 | 0.4 | 0 | 0.9 | 0.5 |