Multi-hop Question Answering on MARCH
73Exact Match (EM)Human
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| HumanModel=N/A2025.09 | 73 | 62 | 67.5 | — | |
| CLARIONModel=Qwen3-235b2025.09 | 38.73 | 28.38 | 33.56 | 3.474 | |
| CLARIONModel=DeepSeek-v3.12025.09 | 31.47 | 27.03 | 29.25 | 3.042 | |
| CLARIONModel=Gemini-2.52025.09 | 29.12 | 26.3 | 27.71 | 2.752 | |
| DIVAModel=Qwen3-235b2025.09 | 28.82 | 22.73 | 25.78 | 3.015 | |
| CoT w/ RAGModel=Qwen3-235b2025.09 | 25.63 | 26.61 | 26.12 | 2.947 | |
| NaiveRAGModel=Qwen3-235b2025.09 | 25.1 | 26.2 | 25.65 | 2.752 | |
| CLARION w/o clarificationModel=Qwen3-235b2025.09 | 25.1 | 25.56 | 25.33 | 2.922 | |
| CLARION w/o clarification & detectionModel=DeepSeek-v3.12025.09 | 24.51 | 24.31 | 24.41 | 2.906 | |
| CLARION w/o clarificationModel=Gemini-2.52025.09 | 24.12 | 22.54 | 23.33 | 2.609 | |
| CLARION w/o clarification & detectionModel=Gemini-2.52025.09 | 23.82 | 22.04 | 22.93 | 2.573 | |
| CLARION w/o clarificationModel=DeepSeek-v3.12025.09 | 23.63 | 22.99 | 23.31 | 2.927 | |
| ReActModel=DeepSeek-v3.12025.09 | 23.17 | 24.78 | 23.97 | 2.723 | |
| CoT w/ RAGModel=Gemini-2.52025.09 | 23.15 | 27.31 | 25.23 | 2.373 | |
| CLARION w/o clarification & detectionModel=Qwen3-235b2025.09 | 22.94 | 24.02 | 23.48 | 2.782 | |
| NaiveRAGModel=Gemini-2.52025.09 | 22.16 | 28.63 | 25.4 | 2.297 | |
| CoTModel=Qwen3-235b2025.09 | 21.51 | 22.32 | 21.91 | 2.897 | |
| CoT w/ RAGModel=DeepSeek-v3.12025.09 | 21.33 | 23.18 | 22.25 | 2.632 | |
| ReActModel=Gemini-2.52025.09 | 21.32 | 22.37 | 21.84 | 2.428 | |
| No RetrievalModel=Qwen3-235b2025.09 | 20.98 | 21.19 | 21.09 | 3.083 | |
| ReActModel=Qwen3-235b2025.09 | 20.98 | 21 | 20.99 | 2.832 | |
| NaiveRAGModel=DeepSeek-v3.12025.09 | 20.2 | 25.03 | 22.62 | 2.084 | |
| CoTModel=DeepSeek-v3.12025.09 | 19.8 | 22.12 | 20.96 | 2.512 | |
| DIVAModel=Gemini-2.52025.09 | 18.82 | 20.29 | 19.56 | 2.303 | |
| DIVAModel=DeepSeek-v3.12025.09 | 18.82 | 20.66 | 19.74 | 2.636 | |
| No RetrievalModel=DeepSeek-v3.12025.09 | 17.75 | 18.72 | 18.24 | 2.683 | |
| CoTModel=Gemini-2.52025.09 | 16.32 | 17.52 | 16.92 | 2.258 | |
| No RetrievalModel=Gemini-2.52025.09 | 15.59 | 20.1 | 17.85 | 2.307 |