Multi-Hop Question Answering on HotpotQA (EM)
50.2Exact Match (EM)Tree-GRPO
Evaluation results
| Method | Links | |
|---|---|---|
| Tree-GRPOBackbone=Qwen2.5-14b2025.09 | 50.2 | |
| GSPOBackbone=Qwen2.5-14b2025.09 | 50.1 | |
| EvalActBackbone=Qwen2.5-7B-Instruct2026.03 | 48.8 | |
| GRPOBackbone=Qwen2.5-14b2025.09 | 47.7 | |
| Agentic-RSearch Agent=R1-Searcher, Search Agent Domain=out-of-domain2026.01 | 47.68 | |
| Agentic-RSearch Agent=Our Search Agent, Search Agent Domain=in-domain2026.01 | 45.82 | |
| AutoRefineBackbone=Qwen2.5-7B-Instruct2026.03 | 45.1 | |
| ReCal+Routing Strategy=RL-based Routing, Task Decomposition=false2026.06 | 44.8 | |
| Tree-GRPOBackbone=Llama3.2-3b2025.09 | 44.6 | |
| Tree-GRPOBackbone=Qwen2.5-7b2025.09 | 44.6 | |
| BGESearch Agent=R1-Searcher, Search Agent Domain=out-of-domain2026.01 | 44.36 | |
| EvalActBackbone=Qwen2.5-3B-Instruct2026.03 | 44.3 | |
| ReCalRouting Strategy=RL-based Routing, Task Decomposition=false2026.06 | 44.2 | |
| SCARLetSearch Agent=R1-Searcher, Search Agent Domain=out-of-domain2026.01 | 44.11 | |
| E5Search Agent=R1-Searcher, Search Agent Domain=out-of-domain2026.01 | 43.56 | |
| REPLUGSearch Agent=Our Search Agent, Search Agent Domain=in-domain2026.01 | 42.63 | |
| GRPOBackbone=Qwen2.5-7b2025.09 | 42.5 | |
| Tree-GRPOBackbone=Qwen2.5-3b2025.09 | 42.4 | |
| SCARLetSearch Agent=Our Search Agent, Search Agent Domain=in-domain2026.01 | 42.34 | |
| BGESearch Agent=Our Search Agent, Search Agent Domain=in-domain2026.01 | 41.71 | |
| E5Search Agent=Our Search Agent, Search Agent Domain=in-domain2026.01 | 41.68 | |
| LLM-EmbedderSearch Agent=R1-Searcher, Search Agent Domain=out-of-domain2026.01 | 41.39 | |
| BAR-RAGBackbone=LLaMA-3.1-8B-Instruct, Number of Iterations=32026.02 | 41.2 | |
| Router-R1Routing Strategy=RL-based Routing, Task Decomposition=false2026.06 | 41.2 | |
| InForage-instructAlg.=PPO, Supervision=SFT + RL (Upper Bound)2026.04 | 40.9 | |
| CoDA-BaseBackbone=Qwen2.5-3B, Retrieval Protocol=w/ Multi-Hop Retrieval2025.12 | 40.7 | |
| REPLUGSearch Agent=R1-Searcher, Search Agent Domain=out-of-domain2026.01 | 40.68 | |
| BAR-RAGBackbone=LLaMA-3.1-8B-Instruct, Number of Iterations=22026.02 | 40.5 | |
| AutoRefineBackbone=Qwen2.5-3B-Instruct2026.03 | 40.5 | |
| GSPOBackbone=Qwen2.5-3b2025.09 | 40.2 | |
| GSPOBackbone=Qwen2.5-7b2025.09 | 40 | |
| Agentic-RSearch Agent=SimpleDeepSearcher, Search Agent Domain=out-of-domain2026.01 | 39.75 | |
| BAR-RAGBackbone=LLaMA-3.1-8B-Instruct, Number of Iterations=12026.02 | 39.5 | |
| LLM-EmbedderSearch Agent=Our Search Agent, Search Agent Domain=in-domain2026.01 | 39.35 | |
| ReActBackbone=Qwen2.5-14b2025.09 | 39.1 | |
| GRPOBackbone=Qwen2.5-3b2025.09 | 39 | |
| BAR-RAGBackbone=Qwen-2.5-7B-Instruct, Number of Iterations=32026.02 | 38.8 | |
| O2-Searcher-instructAlg.=GRPO, Supervision=SFT + RL (Upper Bound)2026.04 | 38.8 | |
| AutoRefine-BaseBackbone=Qwen2.5-3B, Retrieval Protocol=w/ Multi-Hop Retrieval2025.12 | 38.2 | |
| BAR-RAGBackbone=Qwen-2.5-7B-Instruct, Number of Iterations=22026.02 | 38.1 | |
| SCARLetSearch Agent=SimpleDeepSearcher, Search Agent Domain=out-of-domain2026.01 | 37.39 | |
| REPLUGSearch Agent=SimpleDeepSearcher, Search Agent Domain=out-of-domain2026.01 | 37.39 | |
| BGESearch Agent=SimpleDeepSearcher, Search Agent Domain=out-of-domain2026.01 | 37.31 | |
| BAR-RAGBackbone=Qwen-2.5-7B-Instruct, Number of Iterations=12026.02 | 37.2 | |
| Search-R1Backbone=Qwen2.5-7B-Instruct2026.03 | 37 | |
| E5Search Agent=SimpleDeepSearcher, Search Agent Domain=out-of-domain2026.01 | 36.61 | |
| SubSearch-base (Ours)Alg.=GRPO, Supervision=RL (SFT-Free)2026.04 | 36.2 | |
| GRPOBackbone=Llama3.2-3b2025.09 | 36 | |
| RAG SFTBackbone=LLaMA-3.1-8B-Instruct2026.02 | 35.9 | |
| LLM-EmbedderSearch Agent=SimpleDeepSearcher, Search Agent Domain=out-of-domain2026.01 | 35.01 | |
| ZEROSEARCH-instBackbone=Qwen-2.5-7B, Variant=Instruct2025.05 | 34.6 | |
| StepSearch-instructAlg.=PPO, Supervision=RL (SFT-Free)2026.04 | 34.5 | |
| ZEROSEARCH-baseBackbone=Qwen-2.5-3B, Variant=Base2025.05 | 33.8 | |
| BAR-RAGBackbone=Qwen-2.5-3B-Instruct, Number of Iterations=22026.02 | 33.2 | |
| Search-R1-instBackbone=Qwen-2.5-3B, Variant=Instruct2025.05 | 33.2 | |
| Search-R1-instructAlg.=GRPO, Supervision=RL (SFT-Free)2026.04 | 33.1 | |
| StepSearch-baseAlg.=PPO, Supervision=RL (SFT-Free)2026.04 | 32.9 | |
| Search-R1-instBackbone=Qwen-2.5-7B, Variant=Instruct2025.05 | 32.8 | |
| BAR-RAGBackbone=Qwen-2.5-3B-Instruct, Number of Iterations=32026.02 | 32.6 | |
| RAG w/RerankerBackbone=LLaMA-3.1-8B-Instruct2026.02 | 32.5 | |
| RAG SFTBackbone=Qwen-2.5-7B-Instruct2026.02 | 32.4 | |
| Search-R1Backbone=Qwen2.5-3B-Instruct2026.03 | 32.4 | |
| ZEROSEARCH-baseBackbone=LLaMA-3.2-3B, Variant=Base2025.05 | 32.2 | |
| ZEROSEARCH-baseBackbone=Qwen-2.5-7B, Variant=Base2025.05 | 32 | |
| Search-R1-baseBackbone=Qwen-2.5-7B, Variant=Base2025.05 | 31.2 | |
| BAR-RAGBackbone=Qwen-2.5-3B-Instruct, Number of Iterations=12026.02 | 31 | |
| R-Search-instruct (r.)Alg.=GRPO, Supervision=RL (SFT-Free)2026.04 | 30.7 | |
| ReSearch-BaseBackbone=Qwen2.5-3B, Retrieval Protocol=w/ Multi-Hop Retrieval2025.12 | 30.5 | |
| RAGBackbone=LLaMA-3.1-8B-Instruct2026.02 | 30.3 | |
| Query Decomp-base (Ours)Alg.=GRPO, Supervision=RL (SFT-Free)2026.04 | 30.2 | |
| Naïve RAGBackbone=Qwen2.5-7B-Instruct2026.03 | 29.9 | |
| ZeroSearch-baseAlg.=GRPO, Supervision=RL (SFT-Free)2026.04 | 29.8 | |
| Search-R1-BaseBackbone=Qwen2.5-3B, Retrieval Protocol=w/ Multi-Hop Retrieval2025.12 | 29.7 | |
| Search-R1-baseAlg.=GRPO, Supervision=RL (SFT-Free)2026.04 | 29.7 | |
| Search-R1-baseBackbone=LLaMA-3.2-3B, Variant=Base2025.05 | 29.6 | |
| Tree-GRPOBackbone=Qwen2.5-1.5b2025.09 | 29.5 | |
| Search-R1-baseBackbone=Qwen-2.5-3B, Variant=Base2025.05 | 29.2 | |
| RAG SFTBackbone=Qwen-2.5-3B-Instruct2026.02 | 28.9 | |
| RAGBackbone=Qwen-2.5-7B-Instruct2026.02 | 28.9 | |
| RAG w/RerankerBackbone=Qwen-2.5-7B-Instruct2026.02 | 28.1 | |
| GSPOBackbone=Llama3.2-3b2025.09 | 28.1 | |
| ReActBackbone=Qwen2.5-7b2025.09 | 27.9 | |
| Largest LLMRouting Strategy=Heuristic & Discriminative Routing, Task Decomposition=false2026.06 | 27.8 | |
| ZeroSearch-instructAlg.=REINF., Supervision=RL (SFT-Free)2026.04 | 27.4 | |
| ZEROSEARCH-instBackbone=Qwen-2.5-3B, Variant=Instruct2025.05 | 27.4 | |
| Search-o1Backbone=Qwen2.5-14b2025.09 | 26.8 | |
| Prompt LLMRouting Strategy=Heuristic & Discriminative Routing, Task Decomposition=false2026.06 | 26.8 | |
| RAG w/RerankerBackbone=Qwen-2.5-3B-Instruct2026.02 | 26.7 | |
| RAGBackbone=Qwen-2.5-7B2025.05 | 25.8 | |
| RAGBackbone=Qwen-2.5-3B-Instruct2026.02 | 25.5 | |
| Naive RAGBackbone=Qwen2.5-3B, Retrieval Protocol=w/ Single-Hop Retrieval2025.12 | 25.5 | |
| Naïve RAGBackbone=Qwen2.5-3B-Instruct2026.03 | 25.5 | |
| RAGAlg.=–, Supervision=Baselines2026.04 | 25.5 | |
| RouterDCRouting Strategy=Heuristic & Discriminative Routing, Task Decomposition=false2026.06 | 24.4 | |
| RAGBackbone=Qwen-2.5-3B2025.05 | 24.2 | |
| CoTBackbone=LLaMA-3.1-8B-Instruct2026.02 | 24.1 | |
| Search-R1Routing Strategy=No Routing, Task Decomposition=false2026.06 | 23.6 | |
| RAGBackbone=LLaMA-3.2-3B2025.05 | 23.4 | |
| GraphRouterRouting Strategy=Heuristic & Discriminative Routing, Task Decomposition=false2026.06 | 23.4 | |
| R1-baseBackbone=LLaMA-3.2-3B, Variant=Base2025.05 | 22.8 |