Multi-hop Question Answering on MusiQue (EM, F1, EFR)
36.8EMZipRL-8B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ZipRL-8BMethod Strategy=Specialized Agent, Backbone=Qwen3-8B2026.05 | 36.8 | 46.7 | — | |
| ASearcher-14BMethod Strategy=Specialized Agent, Backbone=Qwen2.5-14B2026.05 | 36.6 | 47.6 | — | |
| ZipRL-14BMethod Strategy=Specialized Agent, Backbone=Qwen3-14B2026.05 | 33.6 | 43.4 | — | |
| ASearcher-7BMethod Strategy=Specialized Agent, Backbone=Qwen2.5-7B2026.05 | 32.6 | 42.9 | — | |
| Gemini-3-Pro-ReActMethod Strategy=ReAct, Backbone=Gemini-3-Pro2026.05 | 32.2 | 44.7 | — | |
| ZipRL-4BMethod Strategy=Specialized Agent, Backbone=Qwen3-4B2026.05 | 31.4 | 40.4 | — | |
| ZipRL-7BMethod Strategy=Specialized Agent, Backbone=Qwen2.5-7B2026.05 | 31.2 | 41.4 | — | |
| ZipRL-14BMethod Strategy=Specialized Agent, Backbone=Qwen2.5-14B2026.05 | 30.8 | 40.7 | — | |
| ZipRL-3BMethod Strategy=Specialized Agent, Backbone=Qwen2.5-3B2026.05 | 27.8 | 37.5 | — | |
| LongCat-Flash-ReActMethod Strategy=ReAct, Backbone=LongCat-Flash2026.05 | 24.4 | 37.5 | — | |
| NestBrowse-14BMethod Strategy=Specialized Agent, Backbone=Qwen3-14B2026.05 | 24.4 | 35.2 | — | |
| AgentFold-14BMethod Strategy=Specialized Agent, Backbone=Qwen3-14B2026.05 | 22.2 | 32.9 | — | |
| WebSailor-32BMethod Strategy=Standard, Backbone=WebSailor-32B2026.05 | 22 | 34.6 | — | |
| AgentFold-8BMethod Strategy=Specialized Agent, Backbone=Qwen3-8B2026.05 | 21.6 | 30.1 | — | |
| NestBrowse-8BMethod Strategy=Specialized Agent, Backbone=Qwen3-8B2026.05 | 21.4 | 29.5 | — | |
| LongCat-Flash-SummaryMethod Strategy=Summary, Backbone=LongCat-Flash2026.05 | 20 | 33.2 | — | |
| Qwen3-235B-ReActMethod Strategy=ReAct, Backbone=Qwen3-235B2026.05 | 18.2 | 30.4 | — | |
| Search-R1-3BMethod Strategy=Specialized Agent, Backbone=Qwen2.5-3B2026.05 | 18.2 | 22.4 | — | |
| Qwen3-235B-SummaryMethod Strategy=Summary, Backbone=Qwen3-235B2026.05 | 17.8 | 33.2 | — | |
| PruneRAGBackbone=Qwen3-8B, Retrieval Type=Multi-Retrieval, Time Cost (ms)=22542026.01 | 17.6 | 25.6 | 38.4 | |
| AgentFold-4BMethod Strategy=Specialized Agent, Backbone=Qwen3-4B2026.05 | 16 | 26.5 | — | |
| PruneRAGBackbone=Llama-3.1-8B-Instruct, Retrieval Type=Multi-Retrieval, Time Cost (ms)=4742026.01 | 15.4 | 22.9 | 52 | |
| Gemini-3-Pro-SummaryMethod Strategy=Summary, Backbone=Gemini-3-Pro2026.05 | 14.6 | 27.4 | — | |
| Search-R1-7BMethod Strategy=Specialized Agent, Backbone=Qwen2.5-7B2026.05 | 14.6 | 19 | — | |
| RAG-StarBackbone=Llama-3.1-8B-Instruct, Retrieval Type=Multi-Retrieval, Time Cost (ms)=82162026.01 | 13 | 18.9 | 78.5 | |
| Search-o1Backbone=Qwen3-8B, Retrieval Type=Multi-Retrieval, Time Cost (ms)=48412026.01 | 12.4 | 17.4 | 62.1 | |
| RAG-StarBackbone=Qwen3-8B, Retrieval Type=Multi-Retrieval, Time Cost (ms)=147092026.01 | 12.2 | 19.8 | 84.2 | |
| NestBrowse-4BMethod Strategy=Specialized Agent, Backbone=Qwen3-4B2026.05 | 12 | 21.5 | — | |
| ConTReGenBackbone=Qwen3-8B, Retrieval Type=Multi-Retrieval, Time Cost (ms)=45072026.01 | 11.8 | 17.2 | 60 | |
| GPT-4o-ReActMethod Strategy=ReAct, Backbone=GPT-4o2026.05 | 11.8 | 28.9 | — | |
| Search-R1-14BMethod Strategy=Specialized Agent, Backbone=Qwen2.5-14B2026.05 | 11.8 | 16.6 | — | |
| ProbTreeBackbone=Llama-3.1-8B-Instruct, Retrieval Type=Multi-Retrieval, Time Cost (ms)=38382026.01 | 11 | 17.1 | 91.6 | |
| Standard RAGBackbone=Qwen3-8B, Retrieval Type=Single-Retrieval, Time Cost (ms)=8242026.01 | 10.8 | 16.4 | — | |
| ConTReGenBackbone=Llama-3.1-8B-Instruct, Retrieval Type=Multi-Retrieval, Time Cost (ms)=21562026.01 | 10.4 | 14.6 | 51.6 | |
| ReactBackbone=Qwen3-8B, Retrieval Type=Multi-Retrieval, Time Cost (ms)=44972026.01 | 10.2 | 15.7 | 69.5 | |
| DeepSeek-V3.2-SummaryMethod Strategy=Summary, Backbone=DeepSeek-V3.22026.05 | 10.2 | 25.2 | — | |
| DeepSeek-V3.2-ReActMethod Strategy=ReAct, Backbone=DeepSeek-V3.22026.05 | 9.6 | 17.3 | — | |
| Search-o1Backbone=Llama-3.1-8B-Instruct, Retrieval Type=Multi-Retrieval, Time Cost (ms)=16962026.01 | 9.2 | 13.6 | 75.6 | |
| VanillaBackbone=Qwen3-8B, Retrieval Type=No-Retrieval, Time Cost (ms)=7122026.01 | 9 | 15.3 | — | |
| MemoRAGBackbone=Qwen3-8B, Retrieval Type=Single-Retrieval, Time Cost (ms)=103212026.01 | 9 | 16.1 | — | |
| Self-RAGBackbone=Llama-3.1-8B-Instruct, Retrieval Type=Multi-Retrieval, Time Cost (ms)=32002026.01 | 8.2 | 14.3 | 83.3 | |
| Self-RAGBackbone=Qwen3-8B, Retrieval Type=Multi-Retrieval, Time Cost (ms)=32002026.01 | 8.2 | 14.3 | 85.7 | |
| MemoRAGBackbone=Llama-3.1-8B-Instruct, Retrieval Type=Single-Retrieval, Time Cost (ms)=85662026.01 | 7.8 | 12.3 | — | |
| ReactBackbone=Llama-3.1-8B-Instruct, Retrieval Type=Multi-Retrieval, Time Cost (ms)=15002026.01 | 7.8 | 11.6 | 68.4 | |
| WebSailor-7BMethod Strategy=Specialized Agent, Backbone=Qwen2.5-7B2026.05 | 7.6 | 14.9 | — | |
| Standard RAGBackbone=Llama-3.1-8B-Instruct, Retrieval Type=Single-Retrieval, Time Cost (ms)=1532026.01 | 7.2 | 11.5 | — | |
| VanillaBackbone=Llama-3.1-8B-Instruct, Retrieval Type=No-Retrieval, Time Cost (ms)=562026.01 | 5.4 | 9.1 | — | |
| ProbTreeBackbone=Qwen3-8B, Retrieval Type=Multi-Retrieval, Time Cost (ms)=33002026.01 | 5 | 7.6 | 84.3 | |
| WebSailor-3BMethod Strategy=Specialized Agent, Backbone=Qwen2.5-3B2026.05 | 5 | 12.8 | — | |
| GPT-4o-SummaryMethod Strategy=Summary, Backbone=GPT-4o2026.05 | 3.6 | 18.7 | — |