Multi-hop Question Answering on HotpotQA (val)
79Exact MatchSYMPHONY-L
Evaluation results
| Method | Links | |
|---|---|---|
| SYMPHONY-LBackbone=GPT-4, Oracle Feedback=true, Variant=Large2026.01 | 79 | |
| MASTERBackbone=GPT-4, Oracle Feedback=true2026.01 | 76 | |
| Beam RetrievalBackbone=GPT-4, Oracle Feedback=true2026.01 | 73 | |
| LATSBackbone=GPT-4, Oracle Feedback=true2026.01 | 71 | |
| RAPBackbone=GPT-4, Oracle Feedback=true2026.01 | 60 | |
| SYMPHONY-SBackbone=GPT-4, Oracle Feedback=true, Variant=Small2026.01 | 59 | |
| ToTBackbone=GPT-4, Oracle Feedback=true2026.01 | 55 | |
| ReflexionBackbone=GPT-4, Oracle Feedback=true2026.01 | 51 | |
| Apex-SearcherType=Agentic RAG, Backbone=Qwen2.5-7B-Instruct2026.03 | 40.2 | |
| ReActBackbone=GPT-4, Oracle Feedback=true2026.01 | 39 | |
| StepSearch-instructType=Agentic RAG, Backbone=Qwen2.5-7B-Instruct2026.03 | 38.6 | |
| ReasonRAGType=Agentic RAG, Backbone=Qwen2.5-7B-Instruct2026.03 | 38.4 | |
| CoT-SCBackbone=GPT-4, Oracle Feedback=true2026.01 | 38 | |
| DecEx-RAGType=Agentic RAG, Backbone=Qwen2.5-7B-Instruct2026.03 | 37.7 | |
| Search-R1-InstructType=Agentic RAG, Backbone=Qwen2.5-7B-Instruct2026.03 | 37 | |
| Apex-SearcherType=Agentic RAG, Backbone=Qwen2.5-3B-Instruct2026.03 | 35.6 | |
| ZeroSearch-instructType=Agentic RAG, Backbone=Qwen2.5-7B-Instruct2026.03 | 34.6 | |
| StepSearch-instructType=Agentic RAG, Backbone=Qwen2.5-3B-Instruct2026.03 | 34.5 | |
| CoTBackbone=GPT-4, Oracle Feedback=true2026.01 | 34 | |
| RAGType=Standard RAG, Backbone=Qwen2.5-7B-Instruct2026.03 | 29.9 | |
| Search-R1-InstructType=Agentic RAG, Backbone=Qwen2.5-3B-Instruct2026.03 | 28.4 | |
| ZeroSearch-instructType=Agentic RAG, Backbone=Qwen2.5-3B-Instruct2026.03 | 27.4 | |
| RAGType=Standard RAG, Backbone=Qwen2.5-3B-Instruct2026.03 | 25.5 | |
| Search-o1Type=Agentic RAG, Backbone=Qwen2.5-3B-Instruct2026.03 | 22.1 | |
| Search-o1Type=Agentic RAG, Backbone=Qwen2.5-7B-Instruct2026.03 | 18.7 | |
| Direct InferenceType=Non-Retrieval, Backbone=Qwen2.5-7B-Instruct2026.03 | 18.3 | |
| IRCoTType=Iterative RAG, Backbone=Qwen2.5-3B-Instruct2026.03 | 16.4 | |
| Direct InferenceType=Non-Retrieval, Backbone=Qwen2.5-3B-Instruct2026.03 | 14.9 | |
| IRCoTType=Iterative RAG, Backbone=Qwen2.5-7B-Instruct2026.03 | 13.3 | |
| CoTType=Non-Retrieval, Backbone=Qwen2.5-7B-Instruct2026.03 | 9.2 | |
| CoTType=Non-Retrieval, Backbone=Qwen2.5-3B-Instruct2026.03 | 2.1 |