Multi-hop QA on Bamboogle (EM, F1)
56EMSearch-o1
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Search-o1Reasoning Protocol=Retrieval-augmented Reasoning with Reason-in-Documents, Parameter Count=32B2025.01 | 56 | 67.8 | |
| RAG-QwQ-32BReasoning Protocol=Retrieval-augmented Reasoning, Parameter Count=32B2025.01 | 55.2 | 67.4 | |
| Llama3.3-70BReasoning Protocol=Direct Reasoning (w/o Retrieval), Parameter Count=70B2025.01 | 54.4 | 67.8 | |
| RAgent-Qwen2.5-32BReasoning Protocol=Retrieval-augmented Reasoning, Parameter Count=32B2025.01 | 54.4 | 66.4 | |
| RAG-Qwen2.5-32BReasoning Protocol=Retrieval-augmented Reasoning, Parameter Count=32B2025.01 | 52 | 66 | |
| RAgent-QwQ-32BReasoning Protocol=Retrieval-augmented Reasoning, Parameter Count=32B2025.01 | 52 | 64.7 | |
| Qwen2.5-32BReasoning Protocol=Direct Reasoning (w/o Retrieval), Parameter Count=32B2025.01 | 49.6 | 63.2 | |
| Qwen2.5-72BReasoning Protocol=Direct Reasoning (w/o Retrieval), Parameter Count=72B2025.01 | 47.2 | 61.7 | |
| TaSR-RAGBackbone=Qwen2.5-72B-Instruct2026.03 | 45.6 | 49.1 | |
| StructRAGBackbone=Qwen2.5-72B-Instruct2026.03 | 44.8 | 54.5 | |
| QwQ-32BReasoning Protocol=Direct Reasoning (w/o Retrieval), Parameter Count=32B2025.01 | 38.4 | 53.7 | |
| HippoRAGBackbone=Qwen2.5-72B-Instruct2026.03 | 29.6 | 35.9 | |
| TaSR-RAGBackbone=Qwen2.5-7B-Instruct2026.03 | 25.6 | 28.6 | |
| IRCoTBackbone=Qwen2.5-72B-Instruct2026.03 | 23.2 | 36.6 | |
| CoTBackbone=Qwen2.5-7B-Instruct2026.03 | 23.2 | 20.8 | |
| IRCoTBackbone=Qwen2.5-7B-Instruct2026.03 | 22.4 | 32.1 | |
| RAGBackbone=Qwen2.5-72B-Instruct2026.03 | 21.6 | 34.1 | |
| RAGBackbone=Qwen2.5-7B-Instruct2026.03 | 20.8 | 22.4 | |
| CoTBackbone=Qwen2.5-72B-Instruct2026.03 | 19.6 | 31 | |
| HyperGraphRAGBackbone=Qwen2.5-72B-Instruct2026.03 | 19.2 | 27 | |
| Direct InferenceBackbone=Qwen2.5-72B-Instruct2026.03 | 17.6 | 27.8 | |
| HippoRAGBackbone=Qwen2.5-7B-Instruct2026.03 | 15.2 | 23.7 | |
| StructRAGBackbone=Qwen2.5-7B-Instruct2026.03 | 13.6 | 24.5 | |
| Direct InferenceBackbone=Qwen2.5-7B-Instruct2026.03 | 12 | 17.2 | |
| GraphRAGBackbone=Qwen2.5-7B-Instruct2026.03 | 8 | 14.9 | |
| GraphRAGBackbone=Qwen2.5-72B-Instruct2026.03 | 6.4 | 15.8 | |
| HypergraphRAGBackbone=Qwen2.5-7B-Instruct2026.03 | 3.2 | 3.9 |