Question Answering on Bamboogle (Exact Match (EM) Accuracy (%))
48EM Accuracy (%)ICRL
Evaluation Results
| Method | Links | |
|---|---|---|
| ICRLModel=Qwen2.5-7B2026.03 | 48 | |
| SAPO-3B-InstructRetrieval Strategy=Agent, Backbone Model=Qwen2.5-3B2026.03 | 43.2 | |
| SE-SearchRetrieval Strategy=Agent, Backbone Model=Qwen2.5-3B2026.03 | 42.4 | |
| SAPO-3B-BaseRetrieval Strategy=Agent, Backbone Model=Qwen2.5-3B2026.03 | 42.4 | |
| ParallelSearchModel=Qwen2.5-7B2026.03 | 41.1 | |
| EvolveRBackbone Model=Qwen2.5-7B2025.10 | 39.2 | |
| Search-R1Model=Qwen2.5-7B2026.03 | 36.8 | |
| CriticSearchRetrieval Strategy=Agent, Backbone Model=Qwen2.5-3B2026.03 | 36.8 | |
| Search-R1-instructBackbone Model=Qwen2.5-7B2025.10 | 36.8 | |
| InForageRetrieval Strategy=Agent, Backbone Model=Qwen2.5-3B2026.03 | 36 | |
| Reject SamplingModel=Qwen2.5-7B2026.03 | 35.5 | |
| Rejection SamplingBackbone Model=Qwen2.5-7B2025.10 | 35.5 | |
| O2-SearcherRetrieval Strategy=Agent, Backbone Model=Qwen2.5-3B2026.03 | 34.4 | |
| AutoRefine-BaseRetrieval Strategy=Agent, Backbone Model=Qwen2.5-3B2026.03 | 34.4 | |
| ICRLModel=Qwen2.5-3B2026.03 | 33.6 | |
| AutoRefine-InstructRetrieval Strategy=Agent, Backbone Model=Qwen2.5-3B2026.03 | 33.6 | |
| StepSearch-BaseRetrieval Strategy=Agent, Backbone Model=Qwen2.5-3B2026.03 | 32.8 | |
| EvolveRBackbone Model=Qwen2.5-3B2025.10 | 32.8 | |
| Search-o1Model=Qwen2.5-3B2026.03 | 32 | |
| Search-o1Retrieval Strategy=Agent, Backbone Model=Qwen2.5-3B2026.03 | 32 | |
| Search-o1Backbone Model=Qwen2.5-3B2025.10 | 32 | |
| Search-o1Model=Qwen2.5-7B2026.03 | 29.6 | |
| R1-baseModel=Qwen2.5-7B2026.03 | 29.6 | |
| Search-o1Backbone Model=Qwen2.5-7B2025.10 | 29.6 | |
| R1-baseBackbone Model=Qwen2.5-7B2025.10 | 29.6 | |
| R1-instructModel=Qwen2.5-7B2026.03 | 29.3 | |
| R1-instructBackbone Model=Qwen2.5-7B2025.10 | 29.3 | |
| ZeroSearchModel=Qwen2.5-7B2026.03 | 27.8 | |
| ReSearch-InstructRetrieval Strategy=Agent, Backbone Model=Qwen2.5-3B2026.03 | 26.6 | |
| Search-R1Model=Qwen2.5-3B2026.03 | 26.4 | |
| Search-R1-instructBackbone Model=Qwen2.5-3B2025.10 | 26.4 | |
| IRCoTModel=Qwen2.5-3B2026.03 | 24 | |
| IRCoTRetrieval Strategy=IRCoT, Backbone Model=Qwen2.5-3B2026.03 | 24 | |
| IRCoTBackbone Model=Qwen2.5-3B2025.10 | 24 | |
| CoTModel=Qwen2.5-7B2026.03 | 23.2 | |
| Search-R1-InstructRetrieval Strategy=Agent, Backbone Model=Qwen2.5-3B2026.03 | 23.2 | |
| CoTBackbone Model=Qwen2.5-7B2025.10 | 23.2 | |
| IRCoTModel=Qwen2.5-7B2026.03 | 22.4 | |
| R1-BaseRetrieval Strategy=None, Backbone Model=Qwen2.5-3B2026.03 | 22.4 | |
| R1-baseBackbone Model=Qwen2.5-3B2025.10 | 22.4 | |
| IRCoTBackbone Model=Qwen2.5-7B2025.10 | 22.4 | |
| Reject SamplingModel=Qwen2.5-3B2026.03 | 21 | |
| Rejection SamplingBackbone Model=Qwen2.5-3B2025.10 | 21 | |
| RAGModel=Qwen2.5-7B2026.03 | 20.8 | |
| RAGBackbone Model=Qwen2.5-7B2025.10 | 20.8 | |
| R1-instructModel=Qwen2.5-3B2026.03 | 19.2 | |
| R1-InstructRetrieval Strategy=None, Backbone Model=Qwen2.5-3B2026.03 | 19.2 | |
| R1-instructBackbone Model=Qwen2.5-3B2025.10 | 19.2 | |
| ZeroSearch-BaseRetrieval Strategy=Agent, Backbone Model=Qwen2.5-3B2026.03 | 13.9 | |
| Search-R1-BaseRetrieval Strategy=Agent, Backbone Model=Qwen2.5-3B2026.03 | 12.8 | |
| ReSearch-BaseRetrieval Strategy=Agent, Backbone Model=Qwen2.5-3B2026.03 | 12.8 | |
| DirectModel=Qwen2.5-7B2026.03 | 12 | |
| Direct InferenceBackbone Model=Qwen2.5-7B2025.10 | 12 | |
| SFTModel=Qwen2.5-3B2026.03 | 11.2 | |
| SFTModel=Qwen2.5-7B2026.03 | 11.2 | |
| SFTRetrieval Strategy=None, Backbone Model=Qwen2.5-3B2026.03 | 11.2 | |
| SFTBackbone Model=Qwen2.5-3B2025.10 | 11.2 | |
| SFTBackbone Model=Qwen2.5-7B2025.10 | 11.2 | |
| ZeroSearchModel=Qwen2.5-3B2026.03 | 11.1 | |
| Search-R1-baseBackbone Model=Qwen2.5-3B2025.10 | 8.8 | |
| RAGModel=Qwen2.5-3B2026.03 | 8 | |
| Naive RAGRetrieval Strategy=Naive RAG, Backbone Model=Qwen2.5-3B2026.03 | 8 | |
| RAGBackbone Model=Qwen2.5-3B2025.10 | 8 | |
| DirectModel=Qwen2.5-3B2026.03 | 2.4 | |
| Direct GenerationRetrieval Strategy=None, Backbone Model=Qwen2.5-3B2026.03 | 2.4 | |
| Direct InferenceBackbone Model=Qwen2.5-3B2025.10 | 2.4 | |
| CoTModel=Qwen2.5-3B2026.03 | 0 | |
| CoTBackbone Model=Qwen2.5-3B2025.10 | 0 |