Exact Match Evaluation on Multi-Hop Question Answering on Bamboogle
56Exact MatchEvalAct
Evaluation results
| Method | Links | |
|---|---|---|
| EvalActBackbone=Qwen2.5-7B-Instruct2026.03 | 56 | |
| AutoRefineBackbone=Qwen2.5-7B-Instruct2026.03 | 51.2 | |
| Agentic-RSearch Agent=Our Search Agent, Search Agent Domain=in-domain2026.01 | 48 | |
| EvalActBackbone=Qwen2.5-3B-Instruct2026.03 | 48 | |
| E5Search Agent=Our Search Agent, Search Agent Domain=in-domain2026.01 | 44 | |
| E5Search Agent=R1-Searcher, Search Agent Domain=out-of-domain2026.01 | 44 | |
| BGESearch Agent=SimpleDeepSearcher, Search Agent Domain=out-of-domain2026.01 | 42.4 | |
| TSPOBackbone=Qwen2.5-7b-Instruct2026.01 | 41.6 | |
| REPLUGSearch Agent=Our Search Agent, Search Agent Domain=in-domain2026.01 | 41.6 | |
| REPLUGSearch Agent=R1-Searcher, Search Agent Domain=out-of-domain2026.01 | 41.6 | |
| Agentic-RSearch Agent=R1-Searcher, Search Agent Domain=out-of-domain2026.01 | 41.6 | |
| REPLUGSearch Agent=SimpleDeepSearcher, Search Agent Domain=out-of-domain2026.01 | 41.6 | |
| Agentic-RSearch Agent=SimpleDeepSearcher, Search Agent Domain=out-of-domain2026.01 | 41.6 | |
| SCARLetSearch Agent=Our Search Agent, Search Agent Domain=in-domain2026.01 | 40.8 | |
| LLM-EmbedderSearch Agent=SimpleDeepSearcher, Search Agent Domain=out-of-domain2026.01 | 40.8 | |
| E5Search Agent=SimpleDeepSearcher, Search Agent Domain=out-of-domain2026.01 | 40.8 | |
| StepSearchBackbone=Qwen2.5-7b-Instruct2026.01 | 40 | |
| BAR-RAGBackbone=Qwen-2.5-7B-Instruct, Number of Iterations=32026.02 | 39.6 | |
| BAR-RAGBackbone=Qwen-2.5-7B-Instruct, Number of Iterations=22026.02 | 39.1 | |
| BGESearch Agent=Our Search Agent, Search Agent Domain=in-domain2026.01 | 38.4 | |
| Search-R1Backbone=Qwen2.5-7b-Instruct2026.01 | 36.8 | |
| CoDA-BaseBackbone=Qwen2.5-3B, Retrieval Protocol=w/ Multi-Hop Retrieval2025.12 | 36.8 | |
| Search-R1Backbone=Qwen2.5-7B-Instruct2026.03 | 36.8 | |
| BAR-RAGBackbone=Qwen-2.5-7B-Instruct, Number of Iterations=12026.02 | 36.7 | |
| ReasonRAGBackbone=Qwen2.5-7b-Instruct2026.01 | 36 | |
| LLM-EmbedderSearch Agent=Our Search Agent, Search Agent Domain=in-domain2026.01 | 36 | |
| InForage-instructAlg.=PPO, Supervision=SFT + RL (Upper Bound)2026.04 | 36 | |
| Rejection SamplingBackbone=Qwen2.5-7b-Instruct2026.01 | 35.5 | |
| SCARLetSearch Agent=R1-Searcher, Search Agent Domain=out-of-domain2026.01 | 35.19 | |
| SCARLetSearch Agent=SimpleDeepSearcher, Search Agent Domain=out-of-domain2026.01 | 35.19 | |
| BGESearch Agent=R1-Searcher, Search Agent Domain=out-of-domain2026.01 | 34.4 | |
| AutoRefineBackbone=Qwen2.5-3B-Instruct2026.03 | 34.4 | |
| StepSearch-instructAlg.=PPO, Supervision=RL (SFT-Free)2026.04 | 34.4 | |
| O2-Searcher-instructAlg.=GRPO, Supervision=SFT + RL (Upper Bound)2026.04 | 34.4 | |
| LLM-EmbedderSearch Agent=R1-Searcher, Search Agent Domain=out-of-domain2026.01 | 33.6 | |
| BAR-RAGBackbone=LLaMA-3.1-8B-Instruct, Number of Iterations=32026.02 | 33.3 | |
| TSPOBackbone=Qwen2.5-3b-Instruct2026.01 | 32.8 | |
| BAR-RAGBackbone=LLaMA-3.1-8B-Instruct, Number of Iterations=22026.02 | 32.8 | |
| StepSearch-baseAlg.=PPO, Supervision=RL (SFT-Free)2026.04 | 32.8 | |
| StepSearchBackbone=Qwen2.5-3b-Instruct2026.01 | 32.4 | |
| Search-o1Backbone=Qwen2.5-3b-Instruct2026.01 | 32 | |
| Search-o1Backbone=Qwen2.5-3B, Retrieval Protocol=w/ Multi-Hop Retrieval2025.12 | 32 | |
| AutoRefine-BaseBackbone=Qwen2.5-3B, Retrieval Protocol=w/ Multi-Hop Retrieval2025.12 | 32 | |
| Search-o1Backbone=Qwen2.5-3B-Instruct2026.03 | 32 | |
| Search-o1Alg.=–, Supervision=Baselines2026.04 | 32 | |
| M3POBackbone=Qwen2.5-3B-Instruct, Strategy=Multi-Path Perception Policy Optimization, Context Retrieval=Top-3 documents2025.12 | 31.2 | |
| BAR-RAGBackbone=LLaMA-3.1-8B-Instruct, Number of Iterations=12026.02 | 30.4 | |
| Search-o1Backbone=Qwen2.5-7b-Instruct2026.01 | 29.6 | |
| Search-o1Backbone=Qwen2.5-7B-Instruct, Strategy=Search-o1, Context Retrieval=Top-3 documents2025.12 | 29.6 | |
| HRPOBackbone=Qwen2.5-3B-Instruct, Strategy=Hybrid Reasoning Policy Optimization, Context Retrieval=Top-3 documents2025.12 | 29.6 | |
| Search-o1Backbone=Qwen2.5-7B-Instruct2026.03 | 29.6 | |
| R1-instructBackbone=Qwen2.5-7b-Instruct2026.01 | 29.3 | |
| ZeroSearchBackbone=Qwen2.5-7b-Instruct2026.01 | 27.8 | |
| GRPOBackbone=Qwen2.5-3B-Instruct, Strategy=GRPO, Context Retrieval=Top-3 documents2025.12 | 27.2 | |
| RAG SFTBackbone=Qwen-2.5-7B-Instruct2026.02 | 27.1 | |
| BAR-RAGBackbone=Qwen-2.5-3B-Instruct, Number of Iterations=32026.02 | 26.8 | |
| Search-R1Backbone=Qwen2.5-3b-Instruct2026.01 | 26.4 | |
| Search-R1Backbone=Qwen2.5-3B-Instruct2026.03 | 26.4 | |
| BAR-RAGBackbone=Qwen-2.5-3B-Instruct, Number of Iterations=22026.02 | 26.3 | |
| SubSearch-base (Ours)Alg.=GRPO, Supervision=RL (SFT-Free)2026.04 | 26.3 | |
| IRCoTBackbone=LLaMA-3.1-8B-Instruct2026.02 | 24.5 | |
| RAG SFTBackbone=LLaMA-3.1-8B-Instruct2026.02 | 24.3 | |
| IRCoTBackbone=Qwen2.5-3b-Instruct2026.01 | 24 | |
| PPOBackbone=Qwen2.5-3B-Instruct, Strategy=PPO, Context Retrieval=Top-3 documents2025.12 | 24 | |
| IRCoTBackbone=Qwen-2.5-3B-Instruct2026.02 | 24 | |
| BAR-RAGBackbone=Qwen-2.5-3B-Instruct, Number of Iterations=12026.02 | 24 | |
| IRCoTBackbone=Qwen2.5-3B, Retrieval Protocol=w/ Multi-Hop Retrieval2025.12 | 24 | |
| IRCoTBackbone=Qwen2.5-3B-Instruct2026.03 | 24 | |
| ZeroSearch-baseAlg.=GRPO, Supervision=RL (SFT-Free)2026.04 | 23.6 | |
| CoTBackbone=Qwen2.5-7B-Instruct, Strategy=Chain-of-Thought, Context Retrieval=Top-3 documents2025.12 | 23.2 | |
| M3POBackbone=Qwen2.5-1.5B-Instruct, Strategy=Multi-Path Perception Policy Optimization, Context Retrieval=Top-3 documents2025.12 | 23.2 | |
| CoTBackbone=Qwen-2.5-7B-Instruct2026.02 | 23.2 | |
| Search-R1-instructAlg.=GRPO, Supervision=RL (SFT-Free)2026.04 | 23.2 | |
| IRCoTBackbone=Qwen2.5-7b-Instruct2026.01 | 22.4 | |
| IRCoTBackbone=Qwen2.5-7B-Instruct, Strategy=Interleaved retrieval with CoT, Context Retrieval=Top-3 documents2025.12 | 22.4 | |
| IRCoTBackbone=Qwen-2.5-7B-Instruct2026.02 | 22.4 | |
| R1-BaseBackbone=Qwen2.5-3B, Retrieval Protocol=w/o Retrieval2025.12 | 22.4 | |
| IRCoTBackbone=Qwen2.5-7B-Instruct2026.03 | 22.4 | |
| R1-baseAlg.=PPO, Supervision=RL (SFT-Free)2026.04 | 22.4 | |
| HRPOBackbone=Qwen2.5-1.5B-Instruct, Strategy=Hybrid Reasoning Policy Optimization, Context Retrieval=Top-3 documents2025.12 | 21.6 | |
| Rejection SamplingBackbone=Qwen2.5-3b-Instruct2026.01 | 21 | |
| RAGBackbone=Qwen2.5-7b-Instruct2026.01 | 20.8 | |
| RAGBackbone=Qwen2.5-7B-Instruct, Strategy=Retrieval-Augmented Generation, Context Retrieval=Top-3 documents2025.12 | 20.8 | |
| Naïve RAGBackbone=Qwen2.5-7B-Instruct2026.03 | 20.8 | |
| R-Search-instruct (r.)Alg.=GRPO, Supervision=RL (SFT-Free)2026.04 | 20.8 | |
| RAG w/RerankerBackbone=LLaMA-3.1-8B-Instruct2026.02 | 19.5 | |
| R1-instructBackbone=Qwen2.5-3b-Instruct2026.01 | 19.2 | |
| R1-InstructBackbone=Qwen2.5-3B, Retrieval Protocol=w/o Retrieval2025.12 | 19.2 | |
| RAG w/RerankerBackbone=Qwen-2.5-7B-Instruct2026.02 | 18.7 | |
| PPOBackbone=Qwen2.5-1.5B-Instruct, Strategy=PPO, Context Retrieval=Top-3 documents2025.12 | 18.4 | |
| RAGBackbone=LLaMA-3.1-8B-Instruct2026.02 | 17.6 | |
| CoTBackbone=LLaMA-3.1-8B-Instruct2026.02 | 16.3 | |
| RAGBackbone=Qwen-2.5-7B-Instruct2026.02 | 16 | |
| Query Decomp-base (Ours)Alg.=GRPO, Supervision=RL (SFT-Free)2026.04 | 15.6 | |
| ReasonRAGBackbone=Qwen2.5-3b-Instruct2026.01 | 13.6 | |
| RAG SFTBackbone=Qwen-2.5-3B-Instruct2026.02 | 13.5 | |
| ReSearch-BaseBackbone=Qwen2.5-3B, Retrieval Protocol=w/ Multi-Hop Retrieval2025.12 | 12.8 | |
| Search-R1-BaseBackbone=Qwen2.5-3B, Retrieval Protocol=w/ Multi-Hop Retrieval2025.12 | 12.8 | |
| Search-R1-baseAlg.=GRPO, Supervision=RL (SFT-Free)2026.04 | 12.8 | |
| Direct InferenceBackbone=Qwen2.5-7b-Instruct2026.01 | 12 |