Multi-hop Question Answering on Bamboogle (test)
67.5EMARBOR
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ARBORBackbone=Qwen3-14B2026.06 | 67.5 | 78.6 | 75.8 | |
| DAPOBackbone=Qwen3-14B2026.06 | 65.8 | 74.9 | 74.1 | |
| ARBORBackbone=Qwen3-8B2026.06 | 65.3 | 76.5 | 72.9 | |
| ARBORBackbone=Qwen3-4B2026.06 | 65.1 | 76.2 | 73 | |
| DAPOBackbone=Qwen3-4B2026.06 | 63.7 | 74.2 | 72.7 | |
| DAPOBackbone=Qwen3-8B2026.06 | 61.9 | 73.1 | 70.1 | |
| GRPOBackbone=Qwen3-14B2026.06 | 61.9 | 72.8 | 71.5 | |
| GRPOBackbone=Qwen3-4B2026.06 | 61.6 | 71 | 69.8 | |
| GRPOBackbone=Qwen3-8B2026.06 | 61 | 72.2 | 69.4 | |
| Workflow-R1-SearchMulti-turn=true, Backbone=Qwen2.5-32B-Instruct2026.02 | 57.6 | — | — | |
| ExSearchcategory=Advanced RAG, Backbone=Qwen2.5-7B2026.04 | 51.27 | 56.12 | 63.8 | |
| AFlowMulti-turn=false, Backbone=Qwen2.5-32B-Instruct2026.02 | 50.4 | — | — | |
| CoTMulti-turn=false, Backbone=Qwen2.5-32B-Instruct2026.02 | 48 | — | — | |
| Workflow-R1Multi-turn=true, Backbone=Qwen2.5-32B-Instruct2026.02 | 47.2 | — | — | |
| HERA (Qwen-3-14B)category=Ours2026.04 | 46.46 | 49.01 | 60.53 | |
| TIR PromptingBackbone=Qwen3-4B2026.06 | 46.4 | 61.1 | 55.3 | |
| HERA (Llama-3.1-8)category=Ours2026.04 | 45.22 | 46.6 | 56.31 | |
| AceSearcher-8Bcategory=Advanced RAG2026.04 | 45.12 | 47.21 | 51.91 | |
| SC (CoT×5)Multi-turn=false, Backbone=Qwen2.5-32B-Instruct2026.02 | 44.8 | — | — | |
| MedPromptMulti-turn=false, Backbone=Qwen2.5-32B-Instruct2026.02 | 44.8 | — | — | |
| OThink-SRR1Base model=Qwen2.5-7B-Instruct2026.03 | 44 | — | 53.69 | |
| SE-Search-3BRetrieval=Agent w/ Retrieval2026.02 | 42.4 | — | — | |
| ReSearchBase model=Qwen2.5-7B-Instruct2026.03 | 42.4 | — | 53.65 | |
| TIR PromptingBackbone=Qwen3-14B2026.06 | 42.1 | 49.9 | 50.3 | |
| DirectMulti-turn=false, Backbone=Qwen2.5-32B-Instruct2026.02 | 40.8 | — | — | |
| Search-R1 (GRPO)Multi-turn=true, Backbone=Qwen2.5-32B-Instruct2026.02 | 40 | — | — | |
| Search-R1Base model=Qwen2.5-7B-Instruct2026.03 | 39.2 | — | 49.71 | |
| R1-Searcher 7Bcategory=Advanced RAG2026.04 | 38.85 | 41 | 46.76 | |
| MaASMulti-turn=false, Backbone=Qwen2.5-32B-Instruct2026.02 | 38.4 | — | — | |
| QCompilerStrategy=Query Understanding, Base Model=Llama3.2-3B-Instruct, Fine-tuned=true2025.05 | 37.1 | 41.9 | 49.2 | |
| Search-R1 (PPO)Multi-turn=true, Backbone=Qwen2.5-32B-Instruct2026.02 | 36.8 | — | — | |
| CriticSearchRetrieval=Agent w/ Retrieval2026.02 | 36.8 | — | — | |
| InForageRetrieval=Agent w/ Retrieval2026.02 | 36 | — | — | |
| GPT-4o-minicategory=Single-turn Retrieval2026.04 | 36 | 38.36 | 42.81 | |
| Qwen2.5-14B-Instructcategory=Single-turn Retrieval2026.04 | 34.7 | 27.2 | 34.35 | |
| O2-SearcherRetrieval=Agent w/ Retrieval2026.02 | 34.4 | — | — | |
| AutoRefine-BaseRetrieval=Agent w/ Retrieval2026.02 | 34.4 | — | — | |
| DeepResearcher-7Bcategory=Advanced RAG2026.04 | 34.2 | 37.42 | 44.4 | |
| Iter-RetGencategory=Advanced RAG, Backbone=Qwen2.5-7B2026.04 | 34.12 | 35.33 | 40.65 | |
| AutoRefine-InstructRetrieval=Agent w/ Retrieval2026.02 | 33.6 | — | — | |
| StepSearch-BaseRetrieval=Agent w/ Retrieval2026.02 | 32.8 | — | — | |
| Search-r1-7Bcategory=Advanced RAG2026.04 | 32.64 | 35.79 | 43.93 | |
| Search-o1Retrieval=Agent w/ Retrieval2026.02 | 32 | — | — | |
| MMAO-RAG-8Bcategory=Advanced RAG2026.04 | 31.76 | 34.47 | 42.01 | |
| CORAG-8B (Greedy)category=Advanced RAG2026.04 | 31.61 | 33.03 | 41.25 | |
| OThink-SRR1Base model=Qwen2.5-3B-Instruct2026.03 | 31.2 | — | 43.58 | |
| TIR PromptingBackbone=Qwen3-8B2026.06 | 31.2 | 41.3 | 38.1 | |
| IRCoTcategory=Advanced RAG, Backbone=GPT-4o-mini2026.04 | 30.73 | 34.88 | 33.97 | |
| Qwen3-14Bcategory=Single-turn Retrieval2026.04 | 29.6 | 31.54 | 38.31 | |
| Search-R1Base model=Qwen2.5-3B-Instruct2026.03 | 28 | — | 38.12 | |
| ReSearch-InstructRetrieval=Agent w/ Retrieval2026.02 | 26.6 | — | — | |
| ReSearchBase model=Qwen2.5-3B-Instruct2026.03 | 26.4 | — | 36.45 | |
| RQ-RAGStrategy=Query Understanding, Base Model=Qwen-2.5-7B-Instruct2025.05 | 24.8 | 26.4 | 34.7 | |
| IRCoTRetrieval=Workflow w/ Retrieval2026.02 | 24 | — | — | |
| Plan-RAGcategory=Advanced RAG, Backbone=8b2026.04 | 23.57 | 30.01 | 32.12 | |
| InstructRAG-8Bcategory=Advanced RAG2026.04 | 23.44 | 26.16 | 31.18 | |
| Iter-RetgenStrategy=Iterative RAG, Base Model=Qwen-2.5-7B-Instruct2025.05 | 23.2 | 24 | 31.8 | |
| Search-R1-InstructRetrieval=Agent w/ Retrieval2026.02 | 23.2 | — | — | |
| SELF-RAGcategory=Advanced RAG, Backbone=llama2-7B2026.04 | 22.66 | 25.8 | 30.05 | |
| R1-BaseRetrieval=w/o Retrieval2026.02 | 22.4 | — | — | |
| Qwen3-8Bcategory=Single-turn Retrieval2026.04 | 22.4 | 23.87 | 30.28 | |
| IRCOTBase model=Qwen2.5-7B-Instruct2026.03 | 22.4 | — | 34.54 | |
| DAVID-GRPOBackbone=Qwen2.5-1.5B, Training Budget=Low2026.01 | 22 | — | 25.4 | |
| IR-CoTStrategy=Iterative RAG, Base Model=Qwen-2.5-7B-Instruct2025.05 | 21.6 | 26.4 | 34.2 | |
| IRCOTBase model=Qwen2.5-3B-Instruct2026.03 | 20.8 | — | 31.47 | |
| Basic RAGBase model=Qwen2.5-7B-Instruct2026.03 | 20 | — | 29.34 | |
| Iter-RetGenBase model=Qwen2.5-7B-Instruct2026.03 | 20 | — | 29.32 | |
| R1-InstructRetrieval=w/o Retrieval2026.02 | 19.2 | — | — | |
| Llama3-3.1-8B-instructcategory=Single-turn Retrieval2026.04 | 19.2 | 20.46 | 27.24 | |
| Naive RAGStrategy=Sequential RAG, TopK=5, Base Model=Qwen-2.5-7B-Instruct2025.05 | 18.4 | 21.6 | 28.3 | |
| DAVID-GRPOBackbone=Qwen2.5-1.5B, Training Budget=Low2026.01 | 14.4 | — | 24.2 | |
| ZeroSearch-BaseRetrieval=Agent w/ Retrieval2026.02 | 13.9 | — | — | |
| Naive RAGStrategy=Sequential RAG, TopK=10, Base Model=Qwen-2.5-7B-Instruct2025.05 | 13.6 | 15.2 | 22.1 | |
| Tree-GRPOBackbone=Qwen2.5-1.5B, Training Budget=High2026.01 | 13.6 | — | — | |
| Search-o1Backbone=Qwen2.5-1.5B, Training Budget=High2026.01 | 13 | — | — | |
| Search-R1-BaseRetrieval=Agent w/ Retrieval2026.02 | 12.8 | — | — | |
| ReSearch-BaseRetrieval=Agent w/ Retrieval2026.02 | 12.8 | — | — | |
| Tree-GRPOBackbone=Qwen2.5-1.5B, Training Budget=Low2026.01 | 12 | — | 15.5 | |
| SFTRetrieval=w/o Retrieval2026.02 | 11.2 | — | — | |
| Iter-RetGenBase model=Qwen2.5-3B-Instruct2026.03 | 11.2 | — | 19.09 | |
| No RAGBase model=Qwen2.5-7B-Instruct2026.03 | 10.4 | — | 17.57 | |
| Basic RAGBase model=Qwen2.5-3B-Instruct2026.03 | 9.6 | — | 19.06 | |
| Tree-GRPOBackbone=Llama-3.2-1B, Training Budget=Low2026.01 | 9 | — | 11.9 | |
| Qwen2.5-7B-InstructStrategy=Direct Generation, Base Model=Qwen-2.5-7B-Instruct2025.05 | 8.8 | 14.4 | 16.9 | |
| Direct InferenceBackbone=Qwen2.5-1.5B, Training Budget=No Training2026.01 | 8 | — | — | |
| Search-R1-v0.3Backbone=Qwen2.5-1.5B, Training Budget=Low, Method modifier=w/ retrieval reward2026.01 | 8 | — | 14.3 | |
| DAVID-GRPOBackbone=Llama-3.2-1B, Training Budget=Low2026.01 | 8 | — | 14.5 | |
| Naive RAGRetrieval=Workflow w/ Retrieval2026.02 | 8 | — | — | |
| DAVID-GRPOBackbone=Qwen2.5-0.5B, Training Budget=Low2026.01 | 6 | — | 7 | |
| Tree-GRPOBackbone=Qwen2.5-1.5B, Training Budget=Low2026.01 | 5 | — | 8.2 | |
| DAVID-GRPOBackbone=Qwen2.5-0.5B, Training Budget=Low2026.01 | 4.8 | — | 8.1 | |
| GRPOBackbone=Qwen2.5-1.5B, Training Budget=High2026.01 | 4 | — | — | |
| Tree-GRPOBackbone=Llama-3.2-1B, Training Budget=Low2026.01 | 4 | — | 8.9 | |
| StepSearchBackbone=Llama-3.2-1B, Training Budget=Low2026.01 | 4 | — | 11 | |
| StepSearchBackbone=Qwen2.5-1.5B, Training Budget=Low2026.01 | 4 | — | 6 | |
| Tree-GRPOBackbone=Qwen2.5-0.5B, Training Budget=Low2026.01 | 4 | — | 5.5 | |
| Self-RAGStrategy=Iterative RAG, Base Model=Qwen-2.5-7B-Instruct2025.05 | 3.2 | 16.8 | 13.1 | |
| StepSearchBackbone=Qwen2.5-1.5B, Training Budget=Low2026.01 | 3.2 | — | 8.7 | |
| Tree-GRPOBackbone=Qwen2.5-0.5B, Training Budget=Low2026.01 | 3.2 | — | 5.2 | |
| Search-R1-v0.3Backbone=Qwen2.5-1.5B, Training Budget=Low, Method modifier=w/ retrieval reward2026.01 | 3 | — | 5.5 |