Multi-step Reasoning Question Answering on ConFiQA MR (test)
91.3F1 ScoreProbeRAG
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ProbeRAGCategory=Training-Based, Backbone=Qwen3-8B2025.10 | 91.3 | 89 | |
| ProbeRAGCategory=Training-Based, Backbone=Mistral-7B-v0.32025.10 | 90.8 | 88.2 | |
| ProbeRAGCategory=Training-Based, Backbone=LLaMA-3.1-8B-Instruct2025.10 | 89.7 | 87 | |
| ParamMuteCategory=Training-Based, Backbone=Mistral-7B-v0.32025.10 | 86.4 | 82.1 | |
| CANOECategory=Training-Based, Backbone=Mistral-7B-v0.32025.10 | 84.7 | 81.9 | |
| CANOECategory=Training-Based, Backbone=Qwen3-8B2025.10 | 84.6 | 80.7 | |
| ParamMuteCategory=Training-Based, Backbone=Qwen3-8B2025.10 | 83.2 | 79.4 | |
| CANOECategory=Training-Based, Backbone=LLaMA-3.1-8B-Instruct2025.10 | 80.2 | 72.6 | |
| Context-DPOCategory=Training-Based, Backbone=LLaMA-3.1-8B-Instruct2025.10 | 78.5 | 66.9 | |
| ParamMuteCategory=Training-Based, Backbone=LLaMA-3.1-8B-Instruct2025.10 | 73.2 | 61.2 | |
| AdaCADCategory=Decoding-Based, Backbone=Qwen3-8B2025.10 | 72.5 | 64.3 | |
| COIECDCategory=Decoding-Based, Backbone=Qwen3-8B2025.10 | 71.5 | 63.8 | |
| Opin(Instr)Category=Prompt-Based, Backbone=Qwen3-8B2025.10 | 69.7 | 59.5 | |
| KRECategory=Prompt-Based, Backbone=LLaMA-3.1-8B-Instruct2025.10 | 68.7 | 58.9 | |
| KRECategory=Prompt-Based, Backbone=Qwen3-8B2025.10 | 68.4 | 59 | |
| Full-ContextCategory=Baseline, Backbone=Qwen3-8B2025.10 | 66.1 | 55.1 | |
| Opin(Instr)Category=Prompt-Based, Backbone=LLaMA-3.1-8B-Instruct2025.10 | 65.9 | 54 | |
| Context-DPOCategory=Training-Based, Backbone=Qwen3-8B2025.10 | 65.9 | 55 | |
| CADCategory=Decoding-Based, Backbone=Qwen3-8B2025.10 | 64.8 | 53.3 | |
| FaithfulRAGCategory=Prompt-Based, Backbone=LLaMA-3.1-8B-Instruct2025.10 | 62.4 | 53.1 | |
| FaithfulRAGCategory=Prompt-Based, Backbone=Qwen3-8B2025.10 | 62.3 | 54.2 | |
| KRECategory=Prompt-Based, Backbone=Mistral-7B-v0.32025.10 | 60.9 | 45.3 | |
| Opin(Instr)Category=Prompt-Based, Backbone=Mistral-7B-v0.32025.10 | 57.8 | 52.5 | |
| FaithfulRAGCategory=Prompt-Based, Backbone=Mistral-7B-v0.32025.10 | 56.5 | 42.1 | |
| COIECDCategory=Decoding-Based, Backbone=LLaMA-3.1-8B-Instruct2025.10 | 50.9 | 43.3 | |
| Context-DPOCategory=Training-Based, Backbone=Mistral-7B-v0.32025.10 | 50.9 | 31.9 | |
| Full-ContextCategory=Baseline, Backbone=LLaMA-3.1-8B-Instruct2025.10 | 50.3 | 41.3 | |
| AdaCADCategory=Decoding-Based, Backbone=LLaMA-3.1-8B-Instruct2025.10 | 46.7 | 42.5 | |
| CADCategory=Decoding-Based, Backbone=LLaMA-3.1-8B-Instruct2025.10 | 40 | 31.3 | |
| AdaCADCategory=Decoding-Based, Backbone=Mistral-7B-v0.32025.10 | 39.6 | 29.4 | |
| COIECDCategory=Decoding-Based, Backbone=Mistral-7B-v0.32025.10 | 39.3 | 26.3 | |
| Full-ContextCategory=Baseline, Backbone=Mistral-7B-v0.32025.10 | 37.8 | 21.5 | |
| CADCategory=Decoding-Based, Backbone=Mistral-7B-v0.32025.10 | 27.5 | 12.8 | |
| No-ContextCategory=Baseline, Backbone=Qwen3-8B2025.10 | 8 | 2.8 | |
| No-ContextCategory=Baseline, Backbone=LLaMA-3.1-8B-Instruct2025.10 | 6.1 | 1.9 | |
| No-ContextCategory=Baseline, Backbone=Mistral-7B-v0.32025.10 | 4.9 | 0.5 |