Question Answering on Bamboogle
60EMRAGShaper
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| RAGShaperprotocol=Learning-Based, training_data_size=6.5k2026.01 | 60 | — | — | 72.6 | — | |
| RAGShaperprotocol=Learning-Based, training_data_size=4.5k2026.01 | 58.5 | — | — | 70.3 | — | |
| HL-Dataprotocol=Learning-Based, training_data_size=4.5k2026.01 | 50.4 | — | — | 67.5 | — | |
| SLATEBackbone=Qwen2.5-7B-Base2026.02 | 49.4 | — | — | — | — | |
| PRAISEApproach=Ours2026.04 | 48 | — | — | 56.99 | — | |
| VOIBudget=high, Base Model=Qwen3-32B, Tool-call cap=3, Output-token cap=5002026.05 | 48 | — | — | 62 | — | |
| AutoRefine△Retrieval Model=e5-base-v22026.05 | 48 | — | 4.045 | — | — | |
| DecoupleSearchBackbone=Qwen-2.5-14B-Instruct2025.09 | 47.15 | — | — | 49.77 | — | |
| StepSearchBackbone=Qwen2.5-7B-Base2026.02 | 46.7 | — | — | — | — | |
| ProRAGMethod Category=Reinforcement Learning-based Methods2026.01 | 45.6 | — | — | 56.4 | — | |
| GreedyAgentBackbone=Qwen-2.5-14B-Instruct2025.09 | 45.53 | — | — | 48.18 | — | |
| BATSBudget=high, Base Model=Qwen3-32B, Tool-call cap=3, Output-token cap=5002026.05 | 45 | — | — | 58 | — | |
| Search-R1♢Retrieval Model=jina-embeddings-v5-text-nano2026.05 | 44.8 | — | 4.579 | — | — | |
| R1-SearcherApproach=Agentic Search2026.04 | 44 | — | — | 54.62 | — | |
| Search-R1Method Category=Reinforcement Learning-based Methods2026.01 | 43.2 | — | — | 54.1 | — | |
| SEARCH-R1Backbone=Qwen2.5-7B-Base2026.02 | 43.2 | — | — | — | — | |
| Search-R1♢Retrieval Model=e5-base-v22026.05 | 43.2 | — | 4.709 | — | — | |
| LatentRAG♢Retrieval Model=jina-embeddings-v5-text-nano2026.05 | 43.2 | — | 0.592 | — | — | |
| AutoRefine△Retrieval Model=jina-embeddings-v5-text-nano2026.05 | 43.2 | — | 4.053 | — | — | |
| AutoRefine△Retrieval Model=harrier-oss-v1-270m2026.05 | 43.2 | — | 4.36 | — | — | |
| VOIBudget=upper-mid, Base Model=Qwen3-32B, Tool-call cap=2, Output-token cap=3002026.05 | 43 | — | — | 53 | — | |
| DecoupleSearchBackbone=Qwen-2.5-7B-Instruct2025.09 | 42.28 | — | — | 46.52 | — | |
| GRPO SafeSearchBackbone=Qwen-2.5-7B-Instruct2025.10 | 42.1 | — | — | — | — | |
| BATSBudget=upper-mid, Base Model=Qwen3-32B, Tool-call cap=2, Output-token cap=3002026.05 | 42 | — | — | 54 | — | |
| GRPO Utility-Only Ft.Backbone=Qwen-2.5-7B-Instruct2025.10 | 41.3 | — | — | — | — | |
| Search BlockBackbone=Qwen-2.5-7B-Instruct2025.10 | 40.8 | — | — | — | — | |
| PPO SafeSearchBackbone=Qwen-2.5-7B-Instruct2025.10 | 40.8 | — | — | — | — | |
| LatentRAG△Retrieval Model=e5-base-v22026.05 | 40.8 | — | 0.519 | — | — | |
| LatentRAG△Retrieval Model=jina-embeddings-v5-text-nano2026.05 | 40.8 | — | 0.514 | — | — | |
| AutoRefine△Retrieval Model=F2LLM-v2-330M2026.05 | 40.8 | — | 4.381 | — | — | |
| ITER-RETGENType=Mem.2026.01 | 40 | — | — | — | — | |
| StepSearchType=RL Training2026.01 | 40 | — | — | — | — | |
| LatentRAG♢Retrieval Model=Qwen3-Embedding-0.6B2026.05 | 40 | — | 0.623 | — | — | |
| Search-R1♢Retrieval Model=harrier-oss-v1-270m2026.05 | 40 | — | 4.944 | — | — | |
| BAR-RAGBackbone=Qwen2.5-7B-Instruct2026.02 | 39.6 | — | — | — | — | |
| Document FilterBackbone=Qwen-2.5-7B-Instruct2025.10 | 39.6 | — | — | — | — | |
| HiPRAGMethod Category=Reinforcement Learning-based Methods2026.01 | 39.2 | — | — | 51.6 | — | |
| StepSearchApproach=Process Reward2026.04 | 39.2 | — | — | 50.32 | — | |
| AutoRefine△Retrieval Model=Qwen3-Embedding-0.6B2026.05 | 39.2 | — | 4.224 | — | — | |
| LatentRAG△Retrieval Model=Qwen3-Embedding-0.6B2026.05 | 39.2 | — | 0.581 | — | — | |
| LatentRAG♢Retrieval Model=e5-base-v22026.05 | 39.2 | — | 0.558 | — | — | |
| BAVTBudget=high, Base Model=Qwen3-32B, Tool-call cap=3, Output-token cap=5002026.05 | 39 | — | — | 47 | — | |
| PPO Utility-Only Ft.Backbone=Qwen-2.5-7B-Instruct2025.10 | 38.7 | — | — | — | — | |
| Search-o1Method Category=Advanced Methods2026.01 | 38.4 | — | — | 50.1 | — | |
| Search-R1♢Retrieval Model=Qwen3-Embedding-0.6B2026.05 | 38.4 | — | 4.904 | — | — | |
| BaseBackbone=Llama3.1-8B-Instruct2026.05 | 38.4 | — | — | 45.1 | — | |
| RAgentBackbone=Qwen-2.5-14B-Instruct2025.09 | 37.9 | — | — | 43.28 | — | |
| DecEx-RAGprotocol=Learning-Based2026.01 | 37.6 | — | — | 49.3 | — | |
| PPO SafeSearch w/o hf.Backbone=Qwen-2.5-7B-Instruct2025.10 | 37.6 | — | — | — | — | |
| ReSearchBackbone=Qwen2.5-7B-Base2026.02 | 37.3 | — | — | — | — | |
| PPO SafeSearch w/o qr.Backbone=Qwen-2.5-7B-Instruct2025.10 | 37.3 | — | — | — | — | |
| AFlowBudget=high, Base Model=Qwen3-32B, Tool-call cap=3, Output-token cap=5002026.05 | 37 | — | — | 50 | — | |
| Search-R1Backbone=Qwen2.5-7B-Instruct2026.02 | 36.8 | — | — | — | — | |
| Search-R1Type=RL Training2026.01 | 36.8 | — | — | — | — | |
| TIPSBackbone=Qwen-2.5 Instruct, Model Size=7B2026.03 | 36.8 | — | — | — | — | |
| LatentRAG△Retrieval Model=F2LLM-v2-330M2026.05 | 36.8 | — | 0.552 | — | — | |
| GreedyAgentBackbone=Qwen-2.5-7B-Instruct2025.09 | 36.59 | — | — | 39.03 | — | |
| Search-R1Approach=Agentic Search2026.04 | 36.54 | — | — | 48.43 | — | |
| ACRType=SFT, Reasoner=Qwen-2.5-7B-Instruct2026.01 | 36.36 | — | — | — | — | |
| SLATEBackbone=Qwen2.5-3B-Base, Training Data=NQ+HotpotQA2026.02 | 36.1 | — | — | — | — | |
| ReasonRAGMethod Category=Reinforcement Learning-based Methods2026.01 | 36 | — | — | 45.5 | — | |
| TIPSApproach=Process Reward2026.04 | 36 | — | — | 46.2 | — | |
| Search-R1♢Retrieval Model=F2LLM-v2-330M2026.05 | 36 | — | 5.183 | — | — | |
| LatentRAG♢Retrieval Model=F2LLM-v2-330M2026.05 | 36 | — | 0.602 | — | — | |
| Search-o1Backbone=Qwen-2.5-14B-Instruct2025.09 | 35.48 | — | — | 43.41 | — | |
| PPOBackbone=Qwen-2.5 Instruct, Model Size=7B2026.03 | 35.4 | — | — | — | — | |
| LatentRAG△Retrieval Model=harrier-oss-v1-270m2026.05 | 35.2 | — | 0.559 | — | — | |
| PruneRAGBackbone=Qwen3-8B, Time Cost (ms)=88952026.01 | 34.4 | — | — | 46.7 | — | |
| R1-SearcherRetrieval Model=Qwen3-Embedding-0.6B2026.05 | 34.4 | — | 7.491 | — | — | |
| LatentRAG♢Retrieval Model=harrier-oss-v1-270m2026.05 | 34.4 | — | 0.636 | — | — | |
| VanillaBackbone=Qwen3-8B, Time Cost (ms)=34332026.01 | 33.6 | — | — | 45.3 | — | |
| BAR-RAGBackbone=LLaMA-3.1-8B-Instruct2026.02 | 33.3 | — | — | — | — | |
| CoTBackbone=Qwen-2.5-14B-Instruct2025.09 | 33.06 | — | — | 37.58 | — | |
| AFlowBudget=lower-mid, Base Model=Qwen3-32B, Tool-call cap=2, Output-token cap=2002026.05 | 33 | — | — | 42 | — | |
| VOIBudget=lower-mid, Base Model=Qwen3-32B, Tool-call cap=2, Output-token cap=2002026.05 | 33 | — | — | 42 | — | |
| BAVTBudget=upper-mid, Base Model=Qwen3-32B, Tool-call cap=2, Output-token cap=3002026.05 | 33 | — | — | 42 | — | |
| AFlowBudget=upper-mid, Base Model=Qwen3-32B, Tool-call cap=2, Output-token cap=3002026.05 | 33 | — | — | 42 | — | |
| Search-o1Budget=high, Base Model=Qwen3-32B, Tool-call cap=3, Output-token cap=5002026.05 | 33 | — | — | 44 | — | |
| RAG-StarBackbone=Qwen3-8B, Time Cost (ms)=128222026.01 | 32.8 | — | — | 44.9 | — | |
| StepSearchBackbone=Qwen2.5-3B-Base, Training Data=MuSiQue (19k), Knowledge Base=Wiki-182026.02 | 32.8 | — | — | — | — | |
| ZeroSearchRetrieval Model=Qwen3-Embedding-0.6B2026.05 | 32.8 | — | 4.166 | — | — | |
| PPO SafeSearch w/o qr. & hf.Backbone=Qwen-2.5-7B-Instruct2025.10 | 32.3 | — | — | — | — | |
| Search-o1Backbone=Qwen2.5-3B-Instruct2026.02 | 32 | — | — | — | — | |
| DeepRAGRetrieval Model=Qwen3-Embedding-0.6B2026.05 | 32 | — | 2.848 | — | — | |
| RAG-GDBackbone=Llama-3.1-8B, Retriever=E5, K=52026.05 | 32 | — | — | 42.08 | — | |
| COMPASS(8k+2k)Backbone=Qwen2.5-7B-Instruct2026.05 | 32 | — | — | 42.4 | — | |
| Search-o1Backbone=Qwen2.5-3B-Base2026.02 | 31.6 | — | — | — | — | |
| Search-o1Backbone=Qwen-2.5-7B-Instruct2025.09 | 31.45 | — | — | 35.95 | — | |
| Standard RAGBackbone=Qwen3-8B, Time Cost (ms)=36732026.01 | 31.2 | — | — | 38.1 | — | |
| BaseBackbone=Qwen-2.5-7B-Instruct2025.10 | 30.7 | — | — | — | — | |
| RAgentBackbone=Qwen-2.5-7B-Instruct2025.09 | 30.65 | — | — | 35.01 | — | |
| Search-o1protocol=Prompt-Based2026.01 | 30.4 | — | — | 39.9 | — | |
| Search-R1protocol=Learning-Based2026.01 | 30.4 | — | — | 43.2 | — | |
| IKEAprotocol=Learning-Based2026.01 | 30.4 | — | — | 45.3 | — | |
| ConTReGenBackbone=Qwen3-8B, Time Cost (ms)=188302026.01 | 30.4 | — | — | 39.2 | — | |
| Base adapterBackbone=Llama-3.1-8B, Retriever=E52026.05 | 30.4 | — | — | 40.48 | — | |
| BaseBackbone=Qwen2.5-7B-Instruct2026.05 | 30.4 | — | — | 34.2 | — | |
| Search-o1Backbone=Qwen2.5-7B-Base2026.02 | 30.2 | — | — | — | — | |
| PPO SafeSearchBackbone=Qwen-2.5-3B-Instruct2025.10 | 29.9 | — | — | — | — | |
| Teacher-3BModel Family=Qwen 2.5, Teacher Model=Qwen 2.5 3B2025.08 | 29.8 | — | — | — | — |