Multi-Hop Question Answering on Bamboogle (EM, F1, SD, SE)
54.4Exact Match (EM)SD-Search-Instruct
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| SD-Search-InstructTraining Pipeline=Pure RL, External Teacher=None, Backbone=Qwen2.5-7B2026.05 | 54.4 | — | — | — | |
| SD-Search-BaseTraining Pipeline=Pure RL, External Teacher=None, Backbone=Qwen2.5-7B2026.05 | 53.2 | — | — | — | |
| AutoRefine-BaseTraining Pipeline=Pure RL, External Teacher=None, Backbone=Qwen2.5-7B2026.05 | 51.2 | — | — | — | |
| Thinker-InstructTraining Pipeline=Synthesize → SFT, External Teacher=Qwen2.5-72B, Backbone=Qwen2.5-7B2026.05 | 48 | — | — | — | |
| StepSearch-BaseTraining Pipeline=Pure RL, External Teacher=GPT-4o, Backbone=Qwen2.5-7B2026.05 | 46.7 | — | — | — | |
| Search-R1-BaseTraining Pipeline=Pure RL, External Teacher=None, Backbone=Qwen2.5-7B2026.05 | 45.6 | — | — | — | |
| MR-Search-BaseTraining Pipeline=Pure RL, External Teacher=None, Backbone=Qwen2.5-7B2026.05 | 45.2 | — | — | — | |
| StepSearch-baseBackbone=Qwen2.5-7B2026.04 | 42.7 | 51.9 | 2.07 | 20.7 | |
| AutoSearchBackbone=Qwen2.5-7B2026.04 | 41.9 | 54.9 | 1.53 | 27.4 | |
| Search-R1-baseBackbone=Qwen2.5-7B2026.04 | 41.1 | 53.4 | 2.56 | 16.1 | |
| StepSearch-instructBackbone=Qwen2.5-7B2026.04 | 40.3 | 52.4 | 2.02 | 19.9 | |
| StepSearch-InstructTraining Pipeline=Pure RL, External Teacher=GPT-4o, Backbone=Qwen2.5-7B2026.05 | 40 | — | — | — | |
| Search-R1-instructBackbone=Qwen2.5-7B2026.04 | 37.1 | 49.6 | 2.42 | 15.3 | |
| Search-R1-InstructTraining Pipeline=Pure RL, External Teacher=None, Backbone=Qwen2.5-7B2026.05 | 36.8 | — | — | — | |
| HIPRAG-instructBackbone=Qwen2.5-7B2026.04 | 34.7 | 47.8 | 1.45 | 23.9 | |
| StepSearch-instructBackbone=Qwen2.5-3B2026.04 | 33.9 | 44.7 | 2.07 | 16.4 | |
| ZEROSEARCH-baseBackbone=Qwen-2.5-7B, Variant=Base2025.05 | 33.33 | — | — | — | |
| AutoSearchBackbone=Qwen2.5-3B2026.04 | 33.1 | 44.7 | 1.83 | 18.1 | |
| StepSearch-baseBackbone=Qwen2.5-3B2026.04 | 33 | 41.9 | 3.03 | 10.9 | |
| Search-o1Backbone=LLaMA-3.2-3B2025.05 | 32 | — | — | — | |
| Search-R1-baseBackbone=Qwen-2.5-7B, Variant=Base2025.05 | 30.56 | — | — | — | |
| Search-o1Backbone=Qwen-2.5-7B2025.05 | 30.4 | — | — | — | |
| RAgentBackbone=Qwen-2.5-7B2025.05 | 28 | — | — | — | |
| R1-baseBackbone=Qwen-2.5-7B, Variant=Base2025.05 | 27.78 | — | — | — | |
| ZEROSEARCH-instBackbone=Qwen-2.5-7B, Variant=Instruct2025.05 | 27.78 | — | — | — | |
| Search-R1-instBackbone=LLaMA-3.2-3B, Variant=Instruct2025.05 | 27.78 | — | — | — | |
| RAgentBackbone=LLaMA-3.2-3B2025.05 | 26.4 | — | — | — | |
| Search-R1-instBackbone=Qwen-2.5-7B, Variant=Instruct2025.05 | 26.39 | — | — | — | |
| Search-R1-instructBackbone=Qwen2.5-3B2026.04 | 25 | 36.9 | 1.98 | 12.6 | |
| R1-instructBackbone=Qwen-2.5-7B, Variant=Instruct2025.05 | 25 | — | — | — | |
| CoTBackbone=Qwen-2.5-7B2025.05 | 24 | — | — | — | |
| Search-o1Backbone=Qwen-2.5-3B2025.05 | 22.4 | — | — | — | |
| CoTBackbone=LLaMA-3.2-3B2025.05 | 21.6 | — | — | — | |
| R1-instructBackbone=LLaMA-3.2-3B, Variant=Instruct2025.05 | 20.83 | — | — | — | |
| Search-R1-baseBackbone=LLaMA-3.2-3B, Variant=Base2025.05 | 19.44 | — | — | — | |
| ZEROSEARCH-instBackbone=LLaMA-3.2-3B, Variant=Instruct2025.05 | 18.06 | — | — | — | |
| RAGBackbone=Qwen-2.5-7B2025.05 | 16.8 | — | — | — | |
| Direct AnswerBackbone=Qwen-2.5-3B2025.05 | 16.8 | — | — | — | |
| Search-R1-baseBackbone=Qwen2.5-3B2026.04 | 16.1 | 25.9 | 1.69 | 9.5 | |
| ZEROSEARCH-baseBackbone=LLaMA-3.2-3B, Variant=Base2025.05 | 15.28 | — | — | — | |
| RAGBackbone=Qwen-2.5-3B2025.05 | 15.2 | — | — | — | |
| Direct AnswerBackbone=Qwen-2.5-7B2025.05 | 14.4 | — | — | — | |
| ZEROSEARCH-baseBackbone=Qwen-2.5-3B, Variant=Base2025.05 | 13.89 | — | — | — | |
| RAgentBackbone=Qwen-2.5-3B2025.05 | 13.6 | — | — | — | |
| CoTBackbone=Qwen-2.5-3B2025.05 | 12.8 | — | — | — | |
| Search-R1-baseBackbone=Qwen-2.5-3B, Variant=Base2025.05 | 12.5 | — | — | — | |
| Search-R1-instBackbone=Qwen-2.5-3B, Variant=Instruct2025.05 | 12.5 | — | — | — | |
| RAGBackbone=LLaMA-3.2-3B2025.05 | 11.2 | — | — | — | |
| R1-instructBackbone=Qwen-2.5-3B, Variant=Instruct2025.05 | 11.11 | — | — | — | |
| ZEROSEARCH-instBackbone=Qwen-2.5-3B, Variant=Instruct2025.05 | 11.11 | — | — | — | |
| R1-baseBackbone=LLaMA-3.2-3B, Variant=Base2025.05 | 11.11 | — | — | — | |
| HIPRAG-baseBackbone=Qwen2.5-3B2026.04 | 10.5 | 16.8 | 1.72 | 6.11 | |
| Direct AnswerBackbone=LLaMA-3.2-3B2025.05 | 8 | — | — | — | |
| R1-baseBackbone=Qwen-2.5-3B, Variant=Base2025.05 | 5.56 | — | — | — | |
| HIPRAG-instructBackbone=Qwen2.5-3B2026.04 | 4 | 11.9 | 1.56 | 2.57 |