Question Answering on Bamboogle (EM)
64.1EMGiGPO-Instruct
Evaluation Results
| Method | Links | |
|---|---|---|
| GiGPO-InstructRetrieval Strategy=Multi-Hop Retrieval, Supervision Paradigm=Process-supervision RL2026.05 | 64.1 | |
| PyRAG-RLBackbone=LLaMA-3.1-8B-Instruct2026.05 | 48.3 | |
| Search-E1-InstructRetrieval Strategy=Multi-Hop Retrieval, Supervision Paradigm=Process-supervision RL2026.05 | 46.4 | |
| PyRAG-RLBackbone=Qwen2.5-7B-Instruct2026.05 | 46.1 | |
| M2RModel Family=Mistral-7B-Instruct2026.04 | 46 | |
| ReSearchModel Family=Mistral-7B-Instruct2026.04 | 45.5 | |
| PyRAG-RLBackbone=Qwen3-4B-Instruct2026.05 | 43.2 | |
| ReSearchModel Family=Llama-3.1-8B-Instruct2026.04 | 43.1 | |
| M2RModel Family=Llama-3.1-8B-Instruct2026.04 | 42.9 | |
| SUREModel Family=Mistral-7B-Instruct2026.04 | 42.8 | |
| ReSearchBackbone=Qwen2.5-7B-Instruct2026.05 | 42.4 | |
| PiCABackbone=Qwen2.5-7B Instruct2026.05 | 41.9 | |
| COFTModel Family=Mistral-7B-Instruct2026.04 | 40.3 | |
| StepSearchBackbone=Qwen2.5-7B Instruct2026.05 | 40 | |
| StepSearchBackbone=Qwen2.5-7B-Instruct2026.05 | 40 | |
| SUREModel Family=Llama-3.1-8B-Instruct2026.04 | 37.8 | |
| Search-R1Backbone=Qwen2.5-7B Instruct2026.05 | 36.8 | |
| TIPSBackbone=Qwen2.5-7B Instruct2026.05 | 36.8 | |
| Search-R1Backbone=Qwen2.5-7B-Instruct2026.05 | 36.8 | |
| PiCABackbone=Qwen2.5-3B Instruct2026.05 | 34.7 | |
| StepSearchBackbone=Qwen2.5-3B Instruct2026.05 | 34.4 | |
| AutoRefine-BaseRetrieval Strategy=Multi-Hop Retrieval, Supervision Paradigm=Outcome-reward RL2026.05 | 34.4 | |
| StepSearch-InstructRetrieval Strategy=Multi-Hop Retrieval, Supervision Paradigm=Process-supervision RL2026.05 | 34.4 | |
| AutoRefine-InstructRetrieval Strategy=Multi-Hop Retrieval, Supervision Paradigm=Outcome-reward RL2026.05 | 33.6 | |
| StepSearch-BaseRetrieval Strategy=Multi-Hop Retrieval, Supervision Paradigm=Process-supervision RL2026.05 | 32.8 | |
| COFTModel Family=Llama-3.1-8B-Instruct2026.04 | 32.5 | |
| Search-o1Retrieval Strategy=Multi-Hop Retrieval, Supervision Paradigm=Prompting2026.05 | 32 | |
| RAG-RLBackbone=LLaMA-3.1-8B-Instruct2026.05 | 31.8 | |
| Search-o1Backbone=Qwen2.5-3B Instruct2026.05 | 31.6 | |
| Search-o1Backbone=Qwen2.5-7B Instruct2026.05 | 30.2 | |
| RAG-RLBackbone=Qwen2.5-7B-Instruct2026.05 | 29.6 | |
| RAG-SFTBackbone=LLaMA-3.1-8B-Instruct2026.05 | 29.3 | |
| ZEROSEARCHBackbone=Qwen2.5-7B-Instruct2026.05 | 27.7 | |
| RAG-RLBackbone=Qwen3-4B-Instruct2026.05 | 27.6 | |
| RAG-SFTBackbone=Qwen2.5-7B-Instruct2026.05 | 27.1 | |
| ZeroSearchBackbone=Qwen2.5-7B Instruct2026.05 | 26.7 | |
| ReSearch-InstructRetrieval Strategy=Multi-Hop Retrieval, Supervision Paradigm=Outcome-reward RL2026.05 | 26.6 | |
| Search-R1Backbone=Qwen2.5-3B Instruct2026.05 | 26.4 | |
| RAG-SFTBackbone=Qwen3-4B-Instruct2026.05 | 25.4 | |
| Naive RAGModel Family=Mistral-7B-Instruct2026.04 | 25.4 | |
| IRCoTRetrieval Strategy=Multi-Hop Retrieval, Supervision Paradigm=Prompting2026.05 | 24 | |
| Search-R1-InstructRetrieval Strategy=Multi-Hop Retrieval, Supervision Paradigm=Outcome-reward RL2026.05 | 23.2 | |
| R1-BaseRetrieval Strategy=None2026.05 | 22.4 | |
| RAGBackbone=Qwen2.5-7B Instruct2026.05 | 21.4 | |
| Naive RAGModel Family=Llama-3.1-8B-Instruct2026.04 | 20.9 | |
| TIPSBackbone=Qwen2.5-3B Instruct2026.05 | 20.8 | |
| ZeroSearchBackbone=Qwen2.5-3B Instruct2026.05 | 19.3 | |
| R1-InstructRetrieval Strategy=None2026.05 | 19.2 | |
| Vanilla RAGBackbone=LLaMA-3.1-8B-Instruct2026.05 | 17.6 | |
| Vanilla RAGBackbone=Qwen2.5-7B-Instruct2026.05 | 16 | |
| Vanilla RAGBackbone=Qwen3-4B-Instruct2026.05 | 14.8 | |
| Search-R1-BaseRetrieval Strategy=Multi-Hop Retrieval, Supervision Paradigm=Outcome-reward RL2026.05 | 12.8 | |
| ReSearch-BaseRetrieval Strategy=Multi-Hop Retrieval, Supervision Paradigm=Outcome-reward RL2026.05 | 12.8 | |
| Naive GenerationModel Family=Mistral-7B-Instruct2026.04 | 12.5 | |
| Naive GenerationModel Family=Llama-3.1-8B-Instruct2026.04 | 11.6 | |
| MT-PPOBackbone=Qwen2.5-7B Instruct2026.05 | 11.2 | |
| SFTRetrieval Strategy=None2026.05 | 11.2 | |
| MT-PPOBackbone=Qwen2.5-3B Instruct2026.05 | 8 | |
| Naive RAGRetrieval Strategy=Single-Hop Retrieval2026.05 | 8 | |
| RAGBackbone=Qwen2.5-3B Instruct2026.05 | 7.6 | |
| Direct GenerationRetrieval Strategy=None2026.05 | 2.4 |