Multi-hop Question Answering on Musique
46EMINSES + Router
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| INSES + RouterCategory=Ours2026.03 | 46 | — | — | — | — | — | — | — | 47 | — | — | — | — | |
| SiReRAGCategory=Graph-based2026.03 | 44 | — | — | — | — | — | — | — | 43 | — | — | — | — | |
| ARISEBackbone=Qwen2.5-14B-Instruct, Method Strategy=Search-based2025.04 | 40.5 | 65.87 | — | — | — | — | — | — | — | — | — | — | — | |
| LightRAGCategory=Graph-based2026.03 | 38 | — | — | — | — | — | — | — | 42 | — | — | — | — | |
| SynPlanResearch-R1Model Backbone=Qwen3-8B2026.03 | 35.7 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Auto-RAGBackbone=Qwen2.5-14B-Instruct, Method Strategy=Prompt-based2025.04 | 35.5 | 59.05 | — | — | — | — | — | — | — | — | — | — | — | |
| SynPlanResearch-R1Model Backbone=Qwen3-4B2026.03 | 33.3 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Naïve RAGCategory=Text-based, Top-k=102026.03 | 33 | — | — | — | — | — | — | — | 37 | — | — | — | — | |
| SimpleDeepSearcherModel Backbone=Qwen3-8B2026.03 | 32.3 | — | — | — | — | — | — | — | — | — | — | — | — | |
| RAPTORCategory=Graph-based2026.03 | 32 | — | — | — | — | — | — | — | 35 | — | — | — | — | |
| Naïve RAGCategory=Text-based, Top-k=52026.03 | 31 | — | — | — | — | — | — | — | 29 | — | — | — | — | |
| CARLModel Scale=7B-Inst.2026.05 | 30.4 | — | — | — | — | — | — | — | — | — | 2,511 | — | — | |
| SimpleDeepSearcherModel Backbone=Qwen3-4B2026.03 | 29.8 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Auto-RAGBackbone=Qwen2.5-7B-Instruct, Method Strategy=Prompt-based2025.04 | 29.5 | 57.19 | — | — | — | — | — | — | — | — | — | — | — | |
| RATTBackbone=Llama3.1-8B-Instruct, Method Strategy=Search-based2025.04 | 29.5 | 63.66 | — | — | — | — | — | — | — | — | — | — | — | |
| ARISEBackbone=Qwen2.5-7B-Instruct, Method Strategy=Search-based2025.04 | 29 | 62.26 | — | — | — | — | — | — | — | — | — | — | — | |
| Search-R1Model Backbone=Qwen3-8B2026.03 | 28.8 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Search-R1Model Backbone=Qwen3-4B2026.03 | 26.5 | — | — | — | — | — | — | — | — | — | — | — | — | |
| GraphRAGCategory=Graph-based, Top-k=102026.03 | 26 | — | — | — | — | — | — | — | 36 | — | — | — | — | |
| Tree-GRPOBackbone=Qwen2.5-14b2025.09 | 25.9 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Self-AskBackbone=Qwen2.5-14B-Instruct, Method Strategy=Prompt-based2025.04 | 25 | 58.59 | — | — | — | — | — | — | — | — | — | — | — | |
| IRCoTCategory=Text-based2026.03 | 25 | — | — | — | — | — | — | — | 42 | — | — | — | — | |
| ARISEBackbone=Llama3.1-8B-Instruct, Method Strategy=Search-based2025.04 | 24.5 | 66.38 | — | — | — | — | — | — | — | — | — | — | — | |
| RATTBackbone=Qwen2.5-14B-Instruct, Method Strategy=Search-based2025.04 | 24 | 63.76 | — | — | — | — | — | — | — | — | — | — | — | |
| GLM-4Category=LLM only, Protocol=Few-shot CoT2026.03 | 24 | — | — | — | — | — | — | — | 27 | — | — | — | — | |
| GSPOBackbone=Qwen2.5-14b2025.09 | 23.8 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Critic-Embedtop k=5, Reasoner=Search-R12026.05 | 23.46 | 32.77 | — | — | — | — | — | — | — | — | — | — | — | |
| Rejection SamplingModel Backbone=Qwen3-8B2026.03 | 23.4 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Stella-400Mtop k=5, Reasoner=Search-R12026.05 | 23.21 | 32.49 | — | — | — | — | — | — | — | — | — | — | — | |
| GRPOBackbone=Qwen2.5-14b2025.09 | 23.2 | — | — | — | — | — | — | — | — | — | — | — | — | |
| GraphRAGCategory=Graph-based, Top-k=52026.03 | 23 | — | — | — | — | — | — | — | 24 | — | — | — | — | |
| StepSearchBackbone=Qwen2.5-7b-Instruct2026.01 | 22.6 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Agentic-RSearch Agent=R1-Searcher, Search Agent Domain=out-of-domain2026.01 | 22.54 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Critic-Embedtop k=3, Reasoner=Search-R12026.05 | 22.42 | 31.85 | — | — | — | — | — | — | — | — | — | — | — | |
| Agentic-Rtop k=5, Reasoner=Search-R12026.05 | 22.38 | 31.48 | — | — | — | — | — | — | — | — | — | — | — | |
| Dep-SearchBackbone=Qwen2.5-7B-Instruct2026.01 | 22.2 | — | — | — | — | — | — | — | — | — | — | — | — | |
| MemPO2026.02 | 22.1 | 33.48 | — | — | — | — | 0.39 | 15 | — | — | — | — | — | |
| Rejection SamplingModel Backbone=Qwen3-4B2026.03 | 22.1 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Search-R1 PPOModel Scale=7B-Inst.2026.05 | 22.1 | — | — | — | — | — | — | — | — | — | 2,682 | — | — | |
| Query2DocBackbone=Qwen2.5-14B-Instruct, Method Strategy=Prompt-based2025.04 | 22 | 55.79 | — | — | — | — | — | — | — | — | — | — | — | |
| Verify-and-EditBackbone=Qwen2.5-14B-Instruct, Method Strategy=Prompt-based2025.04 | 22 | 55.3 | — | — | — | — | — | — | — | — | — | — | — | |
| Self-AskBackbone=Qwen2.5-7B-Instruct, Method Strategy=Prompt-based2025.04 | 22 | 57.48 | — | — | — | — | — | — | — | — | — | — | — | |
| CoT-SCBackbone=Llama3.1-8B-Instruct, Method Strategy=Search-based2025.04 | 22 | 59.43 | — | — | — | — | — | — | — | — | — | — | — | |
| Search-o1Model Backbone=Qwen3-8B2026.03 | 21.6 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Verify-and-EditBackbone=Qwen2.5-7B-Instruct, Method Strategy=Prompt-based2025.04 | 21.5 | 54.75 | — | — | — | — | — | — | — | — | — | — | — | |
| Auto-RAGBackbone=Llama3.1-8B-Instruct, Method Strategy=Prompt-based2025.04 | 21.5 | 52.61 | — | — | — | — | — | — | — | — | — | — | — | |
| MT-PPOBackbone=Qwen2.5-7b-Instruct2026.01 | 21.5 | — | — | — | — | — | — | — | — | — | — | — | — | |
| GRPOmemory=none2026.02 | 21.5 | 33.78 | — | — | — | — | 0.86 | 33 | — | — | — | — | — | |
| E5Search Agent=R1-Searcher, Search Agent Domain=out-of-domain2026.01 | 21.39 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Agentic-Rtop k=3, Reasoner=Search-R12026.05 | 21.27 | 30.13 | — | — | — | — | — | — | — | — | — | — | — | |
| CARLModel Scale=3B-Inst.2026.05 | 21.1 | — | — | — | — | — | — | — | — | — | 2,546 | — | — | |
| HyDECategory=Text-based2026.03 | 21 | — | — | — | — | — | — | — | 31 | — | — | — | — | |
| Stella-400Mtop k=3, Reasoner=Search-R12026.05 | 20.85 | 29.88 | — | — | — | — | — | — | — | — | — | — | — | |
| Self-AskBackbone=Llama3.1-8B-Instruct, Method Strategy=Prompt-based2025.04 | 20.5 | 52.14 | — | — | — | — | — | — | — | — | — | — | — | |
| HierSearchBackbone=Qwen2.5-7B-Instruct2026.01 | 20.4 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Search-o1Model Backbone=Qwen3-4B2026.03 | 20.4 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Agentic-RSearch Agent=Our Search Agent, Search Agent Domain=in-domain2026.01 | 20.27 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Tree-GRPOBackbone=Qwen2.5-7b2025.09 | 20.2 | — | — | — | — | — | — | — | — | — | — | — | — | |
| O2-SearcherBackbone=Qwen2.5-7B-Instruct2026.01 | 20 | — | — | — | — | — | — | — | — | — | — | — | — | |
| GSPOBackbone=Qwen2.5-7b2025.09 | 19.2 | — | — | — | — | — | — | — | — | — | — | — | — | |
| GRPOBackbone=Qwen2.5-7b2025.09 | 19.1 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Critic-Embedtop k=1, Reasoner=Search-R12026.05 | 19.07 | 28.21 | — | — | — | — | — | — | — | — | — | — | — | |
| Query2DocBackbone=Llama3.1-8B-Instruct, Method Strategy=Prompt-based2025.04 | 19 | 49.91 | — | — | — | — | — | — | — | — | — | — | — | |
| REPLUGSearch Agent=Our Search Agent, Search Agent Domain=in-domain2026.01 | 18.9 | — | — | — | — | — | — | — | — | — | — | — | — | |
| CoDA-BaseBackbone=Qwen2.5-3B, Retrieval Protocol=w/ Multi-Hop Retrieval2025.12 | 18.7 | — | — | — | — | — | — | — | — | — | — | — | — | |
| R1-SearcherBackbone=Qwen2.5-7B-Instruct2026.01 | 18.6 | — | — | — | — | — | — | — | — | — | — | — | — | |
| SCARLetSearch Agent=R1-Searcher, Search Agent Domain=out-of-domain2026.01 | 18.53 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Search-R1 GRPOModel Scale=7B-Inst.2026.05 | 18.5 | — | — | — | — | — | — | — | — | — | 2,579 | — | — | |
| Stella-400Mtop k=1, Reasoner=Search-R12026.05 | 18.49 | 26.84 | — | — | — | — | — | — | — | — | — | — | — | |
| ZeroSearchBackbone=Qwen2.5-7b-Instruct2026.01 | 18.4 | — | — | — | — | — | — | — | — | — | — | — | — | |
| LLM-EmbedderSearch Agent=R1-Searcher, Search Agent Domain=out-of-domain2026.01 | 18.36 | — | — | — | — | — | — | — | — | — | — | — | — | |
| BGESearch Agent=R1-Searcher, Search Agent Domain=out-of-domain2026.01 | 18.36 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Agentic-Rtop k=1, Reasoner=Search-R12026.05 | 18.33 | 27 | — | — | — | — | — | — | — | — | — | — | — | |
| REPLUGSearch Agent=R1-Searcher, Search Agent Domain=out-of-domain2026.01 | 18.32 | — | — | — | — | — | — | — | — | — | — | — | — | |
| TaSR-RAGBackbone=Qwen2.5-72B-Instruct2026.03 | 18.3 | 24.4 | — | — | — | — | — | — | — | — | — | — | — | |
| RATTBackbone=Qwen2.5-7B-Instruct, Method Strategy=Search-based2025.04 | 18 | 56.58 | — | — | — | — | — | — | — | — | — | — | — | |
| ReSearch2026.02 | 17.8 | 29.63 | — | — | — | — | 0.58 | 29 | — | — | — | — | — | |
| RAGModel Backbone=Qwen3-8B2026.03 | 17.8 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Tree-GRPOBackbone=Qwen2.5-3b2025.09 | 17.8 | — | — | — | — | — | — | — | — | — | — | — | — | |
| E5Search Agent=Our Search Agent, Search Agent Domain=in-domain2026.01 | 17.74 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Tree-GRPOBackbone=Llama3.2-3b2025.09 | 17.6 | — | — | — | — | — | — | — | — | — | — | — | — | |
| StepSearchBackbone=Qwen2.5-3b-Instruct2026.01 | 17.4 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Search-R1Backbone=Qwen2.5-7B-Instruct2026.01 | 17.4 | — | — | — | — | — | — | — | — | — | — | — | — | |
| LLM-EmbedderSearch Agent=Our Search Agent, Search Agent Domain=in-domain2026.01 | 17.33 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Agentic-RSearch Agent=SimpleDeepSearcher, Search Agent Domain=out-of-domain2026.01 | 17.29 | — | — | — | — | — | — | — | — | — | — | — | — | |
| DeepResearcher2026.02 | 17 | 26.55 | — | — | — | — | 2.54 | 44 | — | — | — | — | — | |
| GSPOBackbone=Qwen2.5-3b2025.09 | 17 | — | — | — | — | — | — | — | — | — | — | — | — | |
| AutoRefine-BaseBackbone=Qwen2.5-3B, Retrieval Protocol=w/ Multi-Hop Retrieval2025.12 | 16.9 | — | — | — | — | — | — | — | — | — | — | — | — | |
| ZeroSearchBackbone=Qwen2.5-7B-Instruct2026.01 | 16.8 | — | — | — | — | — | — | — | — | — | — | — | — | |
| RAGModel Backbone=Qwen3-4B2026.03 | 16.6 | — | — | — | — | — | — | — | — | — | — | — | — | |
| SCARLetSearch Agent=Our Search Agent, Search Agent Domain=in-domain2026.01 | 16.38 | — | — | — | — | — | — | — | — | — | — | — | — | |
| MultiSearchBackbone=3B-Base2026.05 | 16.3 | 25.6 | 18.1 | — | — | — | — | — | — | — | — | — | — | |
| BGESearch Agent=Our Search Agent, Search Agent Domain=in-domain2026.01 | 16.21 | — | — | — | — | — | — | — | — | — | — | — | — | |
| CoT-SCBackbone=Qwen2.5-14B-Instruct, Method Strategy=Search-based2025.04 | 16 | 57.46 | — | — | — | — | — | — | — | — | — | — | — | |
| REPLUGSearch Agent=SimpleDeepSearcher, Search Agent Domain=out-of-domain2026.01 | 15.8 | — | — | — | — | — | — | — | — | — | — | — | — | |
| AutoRefine2026.05 | 15.7 | 23.3 | 17.5 | — | — | — | — | — | — | — | — | — | — | |
| E5Search Agent=SimpleDeepSearcher, Search Agent Domain=out-of-domain2026.01 | 15.55 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Self-Ask + Search Engine2022.10 | 15.2 | 27.2 | 19.6 | — | — | — | — | — | — | — | — | — | — | |
| GRPOBackbone=Qwen2.5-3b2025.09 | 15.2 | — | — | — | — | — | — | — | — | — | — | — | — | |
| TSPOBackbone=Qwen2.5-7b-Instruct2026.01 | 15.1 | — | — | — | — | — | — | — | — | — | — | — | — |