Question Answering on Bamboogle (test)
53.6EM (%)ICRL
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ICRLModel=Qwen2.5-14B2026.03 | 53.6 | — | |
| Thinker-InstructTraining Pipeline=Synthesize → SFT, External Teacher=Qwen2.5-72B, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 42.4 | — | |
| Thinker-InstructBase Model=Qwen2.5-3B, Number of Seeds=52026.05 | 42.1 | — | |
| SD-Search-InstructTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 40.4 | — | |
| SD-Search-BaseTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 40.2 | — | |
| SD-Search-BaseBase Model=Qwen2.5-3B, Number of Seeds=52026.05 | 40.2 | — | |
| CoTModel=Qwen2.5-14B2026.03 | 40 | — | |
| SFTBackbone=Qwen-2.5-7B, Category=Tool-integrated Methods2025.12 | 35.5 | — | |
| Search-R1Backbone=Qwen-2.5-7B, Category=Tool-integrated Methods2025.12 | 35.5 | — | |
| AutoRefine-BaseTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 34.4 | — | |
| MR-Search-BaseTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 34.4 | — | |
| StepSearch-InstructTraining Pipeline=Pure RL, External Teacher=GPT-4o, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 34.4 | — | |
| INSPOBackbone=Qwen-2.5-7B, Category=Tool-integrated Methods2025.12 | 33.9 | — | |
| AutoRefine-BaseBase Model=Qwen2.5-3B, Number of Seeds=52026.05 | 33.9 | — | |
| AutoRefine-InstructTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 33.6 | — | |
| R2-Searcher2026.06 | 33.6 | 42.2 | |
| INSPOBackbone=Qwen-2.5-3B, Category=Tool-integrated Methods2025.12 | 33.1 | — | |
| StepSearch-BaseTraining Pipeline=Pure RL, External Teacher=GPT-4o, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 32.8 | — | |
| Search-o1Backbone=Qwen-2.5-3B, Category=Tool-integrated Methods2025.12 | 32 | — | |
| Search-o1Training Pipeline=Prompting, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 32 | — | |
| GRPOBackbone=Qwen-2.5-7B, Category=Model Inference2025.12 | 29.6 | — | |
| Search-o1Backbone=Qwen-2.5-7B, Category=Tool-integrated Methods2025.12 | 29.6 | — | |
| ReSearch-InstructTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 26.6 | — | |
| AutoRefine2026.06 | 26.4 | 34.7 | |
| Search-R12026.06 | 25.6 | 35.4 | |
| IRCoTBackbone=Qwen-2.5-3B, Category=Tool-integrated Methods2025.12 | 24 | — | |
| IRCoTTraining Pipeline=Prompting, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 24 | — | |
| Search-R1-InstructTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 23.2 | — | |
| GRPOBackbone=Qwen-2.5-3B, Category=Model Inference2025.12 | 22.4 | — | |
| IRCoTBackbone=Qwen-2.5-7B, Category=Tool-integrated Methods2025.12 | 22.4 | — | |
| R1-BaseTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/o Retrieval, Backbone=Qwen2.5-3B2026.05 | 22.4 | — | |
| SFTBackbone=Qwen-2.5-3B, Category=Tool-integrated Methods2025.12 | 21 | — | |
| RAGBackbone=Qwen-2.5-7B, Category=Tool-integrated Methods2025.12 | 20.8 | — | |
| Initialsearch=disabled2026.05 | 20 | — | |
| R1-InstructTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/o Retrieval, Backbone=Qwen2.5-3B2026.05 | 19.2 | — | |
| ReSearch2026.06 | 16.8 | 25.9 | |
| DirectModel=Qwen2.5-14B2026.03 | 15.2 | — | |
| ZeroSearch2026.06 | 13.8 | 23.8 | |
| ReSearch-BaseTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 12.8 | — | |
| Search-R1-BaseTraining Pipeline=Pure RL, External Teacher=–, Retrieval Strategy=w/ Multi-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 12.8 | — | |
| Search-o12026.06 | 12.8 | 21.6 | |
| Search-R1Backbone=Qwen-2.5-3B, Category=Tool-integrated Methods2025.12 | 12.6 | — | |
| DirectBackbone=Qwen-2.5-7B, Category=Model Inference2025.12 | 12 | — | |
| IRCOT2026.06 | 12 | 21.8 | |
| SFTBackbone=Qwen-2.5-3B, Category=Model Inference2025.12 | 11.2 | — | |
| SFTBackbone=Qwen-2.5-7B, Category=Model Inference2025.12 | 11.2 | — | |
| SFTTraining Pipeline=SFT, External Teacher=–, Retrieval Strategy=w/o Retrieval, Backbone=Qwen2.5-3B2026.05 | 11.2 | — | |
| Dr. Zero2026.05 | 10.4 | — | |
| EVE-Agent2026.05 | 8.8 | — | |
| RAGBackbone=Qwen-2.5-3B, Category=Tool-integrated Methods2025.12 | 8 | — | |
| Naive RAGTraining Pipeline=Zero-Shot, External Teacher=–, Retrieval Strategy=w/ Single-Hop Retrieval, Backbone=Qwen2.5-3B2026.05 | 8 | — | |
| RAG2026.06 | 7.2 | 11.7 | |
| COT2026.06 | 5.6 | 10.1 | |
| Direct Inference2026.06 | 4.8 | 6.78 | |
| DirectBackbone=Qwen-2.5-3B, Category=Model Inference2025.12 | 2.4 | — | |
| Direct GenerationTraining Pipeline=Zero-Shot, External Teacher=–, Retrieval Strategy=w/o Retrieval, Backbone=Qwen2.5-3B2026.05 | 2.4 | — | |
| Initialsearch=enabled2026.05 | 2.4 | — |