Question Answering on MuSiQue (F1, EM)
50F1 ScoreVOI
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| VOIBudget=high, Base Model=Qwen3-32B, Tool-call cap=3, Output-token cap=5002026.05 | 50 | 40 | |
| BATSBudget=high, Base Model=Qwen3-32B, Tool-call cap=3, Output-token cap=5002026.05 | 48 | 37 | |
| Quest-GNNLLM Generator=Llama-3.3-70B-Instruct2025.10 | 44.93 | 36.65 | |
| BATSBudget=upper-mid, Base Model=Qwen3-32B, Tool-call cap=2, Output-token cap=3002026.05 | 43 | 35 | |
| HippoRAG2LLM Generator=Llama-3.3-70B-Instruct2025.10 | 42.44 | 35.16 | |
| BAVTBudget=high, Base Model=Qwen3-32B, Tool-call cap=3, Output-token cap=5002026.05 | 42 | 34 | |
| GraphRAGLLM Generator=Llama-3.3-70B-Instruct2025.10 | 41.9 | 34.1 | |
| GTE-Qwen2-7BLLM Generator=Llama-3.3-70B-Instruct2025.10 | 40.9 | 33.6 | |
| RAPTORLLM Generator=Llama-3.3-70B-Instruct2025.10 | 40.3 | 32.46 | |
| ToG2LLM Generator=Llama-3.3-70B-Instruct2025.10 | 40.28 | 32.75 | |
| VOIBudget=upper-mid, Base Model=Qwen3-32B, Tool-call cap=2, Output-token cap=3002026.05 | 40 | 33 | |
| NV-Embed-v2LLM Generator=Llama-3.3-70B-Instruct2025.10 | 39.26 | 31.66 | |
| GFM-RAGLLM Generator=Llama-3.3-70B-Instruct2025.10 | 39.06 | 30.05 | |
| G-retrieverLLM Generator=Llama-3.3-70B-Instruct2025.10 | 38.04 | 30.13 | |
| KiRAGLLM Generator=Llama-3.3-70B-Instruct2025.10 | 36.35 | 28.66 | |
| VOIBudget=lower-mid, Base Model=Qwen3-32B, Tool-call cap=2, Output-token cap=2002026.05 | 35 | 28 | |
| Search-o1Budget=high, Base Model=Qwen3-32B, Tool-call cap=3, Output-token cap=5002026.05 | 35 | 29 | |
| Auto-RAGLLM Generator=Llama-3.3-70B-Instruct2025.10 | 34.89 | 27.36 | |
| Self-RAGLLM Generator=Llama-3.3-70B-Instruct2025.10 | 34.75 | 25.83 | |
| GNN-RAGLLM Generator=Llama-3.3-70B-Instruct2025.10 | 34.09 | 28.29 | |
| BAVTBudget=upper-mid, Base Model=Qwen3-32B, Tool-call cap=2, Output-token cap=3002026.05 | 34 | 26 | |
| BAVTBudget=lower-mid, Base Model=Qwen3-32B, Tool-call cap=2, Output-token cap=2002026.05 | 33 | 26 | |
| Contriever+IRCoTLLM Generator=Llama-3.3-70B-Instruct2025.10 | 31.62 | 24.29 | |
| PRAISEApproach=Ours2026.04 | 30.73 | 19.03 | |
| BM25LLM Generator=Llama-3.3-70B-Instruct2025.10 | 28.04 | 20.38 | |
| R1-SearcherApproach=Agentic Search2026.04 | 27.74 | 18.16 | |
| StepSearchApproach=Process Reward2026.04 | 27.5 | 18.16 | |
| No RetrievalLLM Generator=Llama-3.3-70B-Instruct2025.10 | 26.26 | 17.29 | |
| RAG-GDBackbone=Llama-3.1-8B, Retriever=E5, K=52026.05 | 23.47 | 13.2 | |
| RAG-GDBackbone=Llama-3.1-8B, Retriever=BM25, K=52026.05 | 22.61 | 12.54 | |
| Search-R1Approach=Agentic Search2026.04 | 22.5 | 12.7 | |
| Base adapterBackbone=Llama-3.1-8B, Retriever=E52026.05 | 22.07 | 11.83 | |
| Base adapterBackbone=Llama-3.1-8B, Retriever=BM252026.05 | 21.98 | 11.46 | |
| DecoupleSearchBackbone=Qwen-2.5-14B-Instruct2025.09 | 21.82 | 18.38 | |
| Search-o1Backbone=Qwen-2.5-14B-Instruct2025.09 | 20.54 | 16.4 | |
| Search-o1Approach=Agentic Search2026.04 | 20.11 | 11.09 | |
| RAG-GDBackbone=Qwen-2.5-7B, Retriever=BM25, K=52026.05 | 19.73 | 9.26 | |
| GreedyAgentBackbone=Qwen-2.5-14B-Instruct2025.09 | 19.39 | 16.06 | |
| RAG-GDBackbone=Qwen-2.5-7B, Retriever=E5, K=52026.05 | 19.26 | 9.14 | |
| Search-o1Budget=upper-mid, Base Model=Qwen3-32B, Tool-call cap=2, Output-token cap=3002026.05 | 19 | 16 | |
| RAgentBackbone=Qwen-2.5-14B-Instruct2025.09 | 18.72 | 16.2 | |
| TIPSApproach=Process Reward2026.04 | 18.59 | 10.59 | |
| DecoupleSearchBackbone=Qwen-2.5-7B-Instruct2025.09 | 17.73 | 17.2 | |
| GreedyAgentBackbone=Qwen-2.5-7B-Instruct2025.09 | 17.01 | 12.4 | |
| BAVTBudget=low, Base Model=Qwen3-32B, Tool-call cap=1, Output-token cap=1002026.05 | 17 | 11 | |
| Search-o1Backbone=Qwen-2.5-7B-Instruct2025.09 | 16.67 | 12.4 | |
| ReasonRAGApproach=Process Reward2026.04 | 16.37 | 8.94 | |
| VOIBudget=low, Base Model=Qwen3-32B, Tool-call cap=1, Output-token cap=1002026.05 | 16 | 12 | |
| AFlowBudget=high, Base Model=Qwen3-32B, Tool-call cap=3, Output-token cap=5002026.05 | 16 | 11 | |
| Base adapterBackbone=Qwen-2.5-7B, Retriever=E52026.05 | 15.76 | 6.95 | |
| Base adapterBackbone=Qwen-2.5-7B, Retriever=BM252026.05 | 15.46 | 6.41 | |
| CoTBackbone=Qwen-2.5-14B-Instruct2025.09 | 15.3 | 9.4 | |
| RAgentBackbone=Qwen-2.5-7B-Instruct2025.09 | 14.57 | 9 | |
| CoTBackbone=Qwen-2.5-7B-Instruct2025.09 | 13.69 | 7.8 | |
| LightRAGLLM Generator=Llama-3.3-70B-Instruct2025.10 | 13.66 | 6.18 | |
| Vanilla RAGBackbone=Qwen-2.5-7B, Retriever=BM252026.05 | 13.05 | 5.87 | |
| Gen-RetrieveBackbone=Qwen-2.5-7B-Instruct2025.09 | 12.95 | 8.2 | |
| StandardBackbone=Qwen-2.5-14B-Instruct2025.09 | 12.91 | 9.4 | |
| Vanilla RAGBackbone=Qwen-2.5-7B, Retriever=E52026.05 | 12.77 | 5.79 | |
| DirectBackbone=Qwen-2.5-14B-Instruct2025.09 | 12.63 | 6.2 | |
| Vanilla RAGBackbone=Llama-3.1-8B, Retriever=BM252026.05 | 12.44 | 5.75 | |
| Vanilla RAGBackbone=Llama-3.1-8B, Retriever=E52026.05 | 12.25 | 6.04 | |
| Judge-RetrieveBackbone=Qwen-2.5-7B-Instruct2025.09 | 12.18 | 7.6 | |
| Gen-RetrieveBackbone=Qwen-2.5-14B-Instruct2025.09 | 11.7 | 8.4 | |
| StandardBackbone=Qwen-2.5-7B-Instruct2025.09 | 11.57 | 7.2 | |
| Jud-RetrieveBackbone=Qwen-2.5-14B-Instruct2025.09 | 11.49 | 9 | |
| DirectBackbone=Qwen-2.5-7B-Instruct2025.09 | 10.96 | 5.8 | |
| Query OnlyBackbone=Qwen-2.5-7B, Retriever=None2026.05 | 10.57 | 3.8 | |
| Direct LLMApproach=Standard2026.04 | 10.14 | 2.57 | |
| Query OnlyBackbone=Llama-3.1-8B, Retriever=None2026.05 | 9.51 | 3.81 | |
| Naive RAGApproach=Standard2026.04 | 9.32 | 3.52 | |
| IterativeBackbone=Qwen-2.5-14B-Instruct2025.09 | 8.7 | 5.21 | |
| BATSBudget=lower-mid, Base Model=Qwen3-32B, Tool-call cap=2, Output-token cap=2002026.05 | 8 | 6 | |
| AFlowBudget=upper-mid, Base Model=Qwen3-32B, Tool-call cap=2, Output-token cap=3002026.05 | 8 | 6 | |
| IterativeBackbone=Qwen-2.5-7B-Instruct2025.09 | 6.39 | 3.8 | |
| Search-o1Budget=lower-mid, Base Model=Qwen3-32B, Tool-call cap=2, Output-token cap=2002026.05 | 6 | 3 | |
| AFlowBudget=lower-mid, Base Model=Qwen3-32B, Tool-call cap=2, Output-token cap=2002026.05 | 4 | 3 | |
| BATSBudget=low, Base Model=Qwen3-32B, Tool-call cap=1, Output-token cap=1002026.05 | 3 | 2 | |
| Search-o1Budget=low, Base Model=Qwen3-32B, Tool-call cap=1, Output-token cap=1002026.05 | 2 | 0 | |
| AFlowBudget=low, Base Model=Qwen3-32B, Tool-call cap=1, Output-token cap=1002026.05 | 1 | 1 |