Multi-hop Question Answering on Bamboogle (EM, RAG Improvement)
45.5Exact Match (EM)PyRAG
Evaluation results
| Method | Links | ||
|---|---|---|---|
| PyRAGBackbone=Qwen2.5-72B-Instruct2026.05 | 45.5 | 23.9 | |
| PyRAGBackbone=Qwen2.5-7B-Instruct2026.05 | 41.5 | 25.5 | |
| ITER-RETGENBackbone=Qwen2.5-72B-Instruct2026.05 | 33.6 | — | |
| ITER-RETGENBackbone=Qwen2.5-7B-Instruct2026.05 | 29.6 | — | |
| IRCoTBackbone=Qwen2.5-72B-Instruct2026.05 | 28.3 | — | |
| Self-AskBackbone=Qwen2.5-72B-Instruct2026.05 | 26.1 | — | |
| IRCoTBackbone=Qwen2.5-7B-Instruct2026.05 | 24.3 | — | |
| CoTBackbone=Qwen2.5-7B-Instruct2026.05 | 23.2 | — | |
| Self-AskBackbone=Qwen2.5-7B-Instruct2026.05 | 22.1 | — | |
| Vanilla RAGBackbone=Qwen2.5-72B-Instruct2026.05 | 21.6 | — | |
| CoTBackbone=Qwen2.5-72B-Instruct2026.05 | 19.6 | — | |
| Direct InferenceBackbone=Qwen2.5-72B-Instruct2026.05 | 17.6 | — | |
| Vanilla RAGBackbone=Qwen2.5-7B-Instruct2026.05 | 16 | — | |
| Direct InferenceBackbone=Qwen2.5-7B-Instruct2026.05 | 12 | — |