Multi-Hop Question Answering on MuSiQue (EM)
25.3Exact Match (EM)EvalAct
Evaluation results
| Method | Links | |
|---|---|---|
| EvalActBackbone=Qwen2.5-7B-Instruct2026.03 | 25.3 | |
| EvalActBackbone=Qwen2.5-3B-Instruct2026.03 | 21.6 | |
| AutoRefineBackbone=Qwen2.5-7B-Instruct2026.03 | 18.7 | |
| StepSearch-baseAlg.=PPO, Supervision=RL (SFT-Free)2026.04 | 18.1 | |
| StepSearch-instructAlg.=PPO, Supervision=RL (SFT-Free)2026.04 | 17.4 | |
| InForage-instructAlg.=PPO, Supervision=SFT + RL (Upper Bound)2026.04 | 17.2 | |
| O2-Searcher-instructAlg.=GRPO, Supervision=SFT + RL (Upper Bound)2026.04 | 16 | |
| AutoRefineBackbone=Qwen2.5-3B-Instruct2026.03 | 15.7 | |
| Search-R1Backbone=Qwen2.5-7B-Instruct2026.03 | 14.6 | |
| BAR-RAGBackbone=LLaMA-3.1-8B-Instruct, Number of Iterations=22026.02 | 12.5 | |
| Search-R1-instructAlg.=GRPO, Supervision=RL (SFT-Free)2026.04 | 12.4 | |
| ZeroSearch-baseAlg.=GRPO, Supervision=RL (SFT-Free)2026.04 | 12.4 | |
| BAR-RAGBackbone=LLaMA-3.1-8B-Instruct, Number of Iterations=32026.02 | 12 | |
| R-Search-instruct (r.)Alg.=GRPO, Supervision=RL (SFT-Free)2026.04 | 11.9 | |
| BAR-RAGBackbone=LLaMA-3.1-8B-Instruct, Number of Iterations=12026.02 | 11.1 | |
| Search-R1Backbone=Qwen2.5-3B-Instruct2026.03 | 10.3 | |
| SubSearch-base (Ours)Alg.=GRPO, Supervision=RL (SFT-Free)2026.04 | 10.1 | |
| Query Decomp-base (Ours)Alg.=GRPO, Supervision=RL (SFT-Free)2026.04 | 9.9 | |
| ZeroSearch-instructAlg.=REINF., Supervision=RL (SFT-Free)2026.04 | 9.8 | |
| BAR-RAGBackbone=Qwen-2.5-7B-Instruct, Number of Iterations=22026.02 | 9.1 | |
| BAR-RAGBackbone=Qwen-2.5-7B-Instruct, Number of Iterations=32026.02 | 9.1 | |
| RAG SFTBackbone=LLaMA-3.1-8B-Instruct2026.02 | 8.9 | |
| BAR-RAGBackbone=Qwen-2.5-7B-Instruct, Number of Iterations=12026.02 | 8.3 | |
| RAG w/RerankerBackbone=LLaMA-3.1-8B-Instruct2026.02 | 7.6 | |
| BAR-RAGBackbone=Qwen-2.5-3B-Instruct, Number of Iterations=22026.02 | 7.4 | |
| BAR-RAGBackbone=Qwen-2.5-3B-Instruct, Number of Iterations=32026.02 | 7.2 | |
| IRCoTBackbone=Qwen-2.5-7B-Instruct2026.02 | 7.2 | |
| IRCoTBackbone=Qwen2.5-7B-Instruct2026.03 | 7.2 | |
| IRCoTBackbone=LLaMA-3.1-8B-Instruct2026.02 | 6.9 | |
| RAG SFTBackbone=Qwen-2.5-7B-Instruct2026.02 | 6.8 | |
| CoTBackbone=LLaMA-3.1-8B-Instruct2026.02 | 6.8 | |
| IRCoTBackbone=Qwen-2.5-3B-Instruct2026.02 | 6.7 | |
| IRCoTBackbone=Qwen2.5-3B-Instruct2026.03 | 6.7 | |
| Search-R1-baseAlg.=GRPO, Supervision=RL (SFT-Free)2026.04 | 6.6 | |
| RAG SFTBackbone=Qwen-2.5-3B-Instruct2026.02 | 6.4 | |
| BAR-RAGBackbone=Qwen-2.5-3B-Instruct, Number of Iterations=12026.02 | 6.3 | |
| RAGBackbone=LLaMA-3.1-8B-Instruct2026.02 | 6.3 | |
| Search-o1Backbone=Qwen2.5-7B-Instruct2026.03 | 5.8 | |
| Naïve RAGBackbone=Qwen2.5-7B-Instruct2026.03 | 5.8 | |
| RAG w/RerankerBackbone=Qwen-2.5-7B-Instruct2026.02 | 5.5 | |
| R1-baseAlg.=PPO, Supervision=RL (SFT-Free)2026.04 | 5.5 | |
| Search-o1Backbone=Qwen2.5-3B-Instruct2026.03 | 5.4 | |
| Search-o1Alg.=–, Supervision=Baselines2026.04 | 5.4 | |
| RAG w/RerankerBackbone=Qwen-2.5-3B-Instruct2026.02 | 4.9 | |
| RAGBackbone=Qwen-2.5-3B-Instruct2026.02 | 4.7 | |
| RAGBackbone=Qwen-2.5-7B-Instruct2026.02 | 4.7 | |
| Naïve RAGBackbone=Qwen2.5-3B-Instruct2026.03 | 4.7 | |
| RAGAlg.=–, Supervision=Baselines2026.04 | 4.7 | |
| SFTAlg.=–, Supervision=Baselines2026.04 | 4.4 | |
| Direct InferenceBackbone=Qwen-2.5-7B-Instruct2026.02 | 3.1 | |
| Direct GenerationBackbone=Qwen2.5-7B-Instruct2026.03 | 3.1 | |
| Direct InferenceBackbone=LLaMA-3.1-8B-Instruct2026.02 | 2.7 | |
| CoTBackbone=Qwen-2.5-7B-Instruct2026.02 | 2.2 | |
| Direct InferenceBackbone=Qwen-2.5-3B-Instruct2026.02 | 2 | |
| Direct GenerationBackbone=Qwen2.5-3B-Instruct2026.03 | 2 | |
| Direct InferenceAlg.=–, Supervision=Baselines2026.04 | 2 | |
| CoTBackbone=Qwen-2.5-3B-Instruct2026.02 | 0.2 | |
| CoTAlg.=–, Supervision=Baselines2026.04 | 0.2 |