Question Answering on 2WikiMultiHopQA (F1, LM, Time)
43Exact MatchTIPS
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| TIPSBackbone=Qwen2.5-7B Instruct2026.05 | 43 | — | — | — | — | — | |
| All-pool PortfolioModel=Llama-3.1-70B-Instruct, k=4, l=32026.05 | 41.87 | — | — | — | — | — | |
| All-pool PortfolioModel=Gemma-3-27B-It, k=4, l=32026.05 | 41.42 | — | — | — | — | — | |
| All-pool PortfolioModel=Llama-3.1-70B-Instruct, k=4, l=22026.05 | 41.39 | — | — | — | — | — | |
| PiCABackbone=Qwen2.5-3B Instruct2026.05 | 40.8 | — | — | — | — | — | |
| All-pool PortfolioModel=Gemma-3-27B-It, k=4, l=22026.05 | 40.52 | — | — | — | — | — | |
| Search-R1Backbone=Qwen2.5-7B Instruct2026.05 | 40.1 | — | — | — | — | — | |
| PiCABackbone=Qwen2.5-7B Instruct2026.05 | 40.1 | — | — | — | — | — | |
| StepSearchBackbone=Qwen2.5-7B Instruct2026.05 | 36.6 | — | — | — | — | — | |
| Best Vendi RetrieverModel=Gemma-3-27B-It2026.05 | 35.62 | — | — | — | — | — | |
| Best DS RetrieverModel=Gemma-3-27B-It2026.05 | 35.44 | — | — | — | — | — | |
| RAG-GDBackbone=Llama-3.1-8B, Retriever=BM25, K=52026.05 | 34.15 | — | 40.04 | — | — | — | |
| Best GraphDense RetrieverModel=Gemma-3-27B-It2026.05 | 34.07 | — | — | — | — | — | |
| RAG-GDBackbone=Llama-3.1-8B, Retriever=E5, K=52026.05 | 33.94 | — | 39.83 | — | — | — | |
| Base adapterBackbone=Llama-3.1-8B, Retriever=BM252026.05 | 33.66 | — | 39.55 | — | — | — | |
| Base adapterBackbone=Llama-3.1-8B, Retriever=E52026.05 | 33.45 | — | 39.45 | — | — | — | |
| StepSearchBackbone=Qwen2.5-3B Instruct2026.05 | 32 | — | — | — | — | — | |
| Search-R1Backbone=Qwen2.5-3B Instruct2026.05 | 31.9 | — | — | — | — | — | |
| ZeroSearchBackbone=Qwen2.5-7B Instruct2026.05 | 30.9 | — | — | — | — | — | |
| Vendi-Portfolio (MPNet)Model=Llama-3.1-70B-Instruct, k=5, l=22026.05 | 30.8 | — | — | — | — | — | |
| RAG-GDBackbone=Qwen-2.5-7B, Retriever=E5, K=52026.05 | 29.67 | — | 35.47 | — | — | — | |
| TIPSBackbone=Qwen2.5-3B Instruct2026.05 | 29.3 | — | — | — | — | — | |
| Nearest-neighbor retrieval (MPNet)Model=Llama-3.1-70B-Instruct2026.05 | 29.21 | — | — | — | — | — | |
| Vendi-RAG Adaptive (T=20) (MPNet)Model=Llama-3.1-70B-Instruct, T=202026.05 | 29 | — | — | — | — | — | |
| Vendi-RAG Adaptive (T=10) (MPNet)Model=Llama-3.1-70B-Instruct, T=102026.05 | 28.9 | — | — | — | — | — | |
| RAG-GDBackbone=Qwen-2.5-7B, Retriever=BM25, K=52026.05 | 28.86 | — | 34.48 | — | — | — | |
| Base adapterBackbone=Qwen-2.5-7B, Retriever=BM252026.05 | 28.22 | — | 34.13 | — | — | — | |
| Base adapterBackbone=Qwen-2.5-7B, Retriever=E52026.05 | 27.89 | — | 33.97 | — | — | — | |
| Vanilla RAGBackbone=Qwen-2.5-7B, Retriever=BM252026.05 | 27.87 | — | 33.24 | — | — | — | |
| Query OnlyBackbone=Llama-3.1-8B, Retriever=None2026.05 | 26.39 | — | 31.04 | — | — | — | |
| Vanilla RAGBackbone=Llama-3.1-8B, Retriever=BM252026.05 | 26.07 | — | 31.82 | — | — | — | |
| Vendi-RAG Adaptive (T=10) (MPNet)Model=Gemma-3-27B-It, T=102026.05 | 25.9 | — | — | — | — | — | |
| Vendi-RAG Adaptive (T=20) (MPNet)Model=Gemma-3-27B-It, T=202026.05 | 25.6 | — | — | — | — | — | |
| Vanilla RAGBackbone=Qwen-2.5-7B, Retriever=E52026.05 | 25.48 | — | 31.43 | — | — | — | |
| Best Vendi RetrieverModel=Llama-3.1-70B-Instruct2026.05 | 24.45 | — | — | — | — | — | |
| Best DS RetrieverModel=Llama-3.1-70B-Instruct2026.05 | 24.43 | — | — | — | — | — | |
| Nearest-neighbor retrieval (MPNet)Model=Gemma-3-27B-It2026.05 | 24.08 | — | — | — | — | — | |
| Query OnlyBackbone=Qwen-2.5-7B, Retriever=None2026.05 | 23.91 | — | 28.12 | — | — | — | |
| ZeroSearchBackbone=Qwen2.5-3B Instruct2026.05 | 23.9 | — | — | — | — | — | |
| Best GraphDense RetrieverModel=Llama-3.1-70B-Instruct2026.05 | 23.87 | — | — | — | — | — | |
| Vanilla RAGBackbone=Llama-3.1-8B, Retriever=E52026.05 | 23.44 | — | 29.46 | — | — | — | |
| Vendi-Portfolio (MPNet)Model=Gemma-3-27B-It, k=5, l=22026.05 | 23.41 | — | — | — | — | — | |
| RAGBackbone=Qwen2.5-7B Instruct2026.05 | 23.1 | — | — | — | — | — | |
| MT-PPOBackbone=Qwen2.5-7B Instruct2026.05 | 22.8 | — | — | — | — | — | |
| No retrievalModel=Gemma-3-27B-It2026.05 | 22.64 | — | — | — | — | — | |
| RAGBackbone=Qwen2.5-3B Instruct2026.05 | 22.1 | — | — | — | — | — | |
| MT-PPOBackbone=Qwen2.5-3B Instruct2026.05 | 21.4 | — | — | — | — | — | |
| Search-o1Backbone=Qwen2.5-3B Instruct2026.05 | 20.7 | — | — | — | — | — | |
| No retrievalModel=Llama-3.1-70B-Instruct2026.05 | 19.2 | — | — | — | — | — | |
| Search-o1Backbone=Qwen2.5-7B Instruct2026.05 | 18.1 | — | — | — | — | — | |
| ChameleonModel=Qwen-Plus2025.01 | — | 56.68 | 59.18 | 60.12 | 1.7 | 37.53 | |
| ChameleonModel=LLaMA-3.1-8B2025.01 | — | 40.52 | 43.49 | 40.27 | 1.84 | 26.71 | |
| GoldRouterModel=Qwen-Plus2025.01 | — | 70.11 | 72.37 | 76.33 | 1.36 | 30.93 | |
| GoldRouterModel=LLaMA-3.1-8B2025.01 | — | 49.73 | 52.99 | 52.07 | 1.58 | 20.57 | |
| HiPRAGBackbone=Qwen2.5-7B2025.10 | — | — | — | 42.8 | — | — | |
| MT-PPOBackbone=Qwen2.5-3B Instruct2026.05 | — | — | 26.1 | — | — | — | |
| MT-PPOBackbone=Qwen2.5-7B Instruct2026.05 | — | — | 28.6 | — | — | — | |
| PiCABackbone=Qwen2.5-3B Instruct2026.05 | — | — | 47.2 | — | — | — | |
| PiCABackbone=Qwen2.5-7B Instruct2026.05 | — | — | 46.6 | — | — | — | |
| RAGBackbone=Qwen2.5-3B Instruct2026.05 | — | — | 31.6 | — | — | — | |
| RAGBackbone=Qwen2.5-7B Instruct2026.05 | — | — | 22.6 | — | — | — | |
| ReSeekBackbone=Qwen2.5-7B2025.10 | — | — | — | 38.2 | — | — | |
| RIRSModel=Qwen-Plus2025.01 | — | 61.14 | 63.71 | 65.78 | 1.73 | 39.76 | |
| RIRSModel=LLaMA-3.1-8B2025.01 | — | 45.65 | 49.1 | 45.65 | 2.02 | 22.26 | |
| RIRSModel=Mixed, Note=RAG agents use LLaMA-3.1-8B, other modules use Qwen-Plus2025.01 | — | 59.48 | 61.94 | 62.06 | 1.85 | 39.37 | |
| RouterDCModel=Qwen-Plus2025.01 | — | 50.97 | 53.88 | 52.12 | 1.67 | 45.61 | |
| RouterDCModel=LLaMA-3.1-8B2025.01 | — | 34.48 | 37.35 | 30.59 | 1.67 | 26.58 | |
| Search-o1Backbone=Qwen2.5-3B Instruct2026.05 | — | — | 30.9 | — | — | — | |
| Search-o1Backbone=Qwen2.5-7B Instruct2026.05 | — | — | 28.9 | — | — | — | |
| Search-R1Backbone=Qwen2.5-3B Instruct2026.05 | — | — | 35.2 | — | — | — | |
| Search-R1Backbone=Qwen2.5-7B Instruct2026.05 | — | — | 46.5 | — | — | — | |
| StepSearchBackbone=Qwen2.5-3B Instruct2026.05 | — | — | 38.5 | — | — | — | |
| StepSearchBackbone=Qwen2.5-7B Instruct2026.05 | — | — | 43.1 | — | — | — | |
| TIPSBackbone=Qwen2.5-3B Instruct2026.05 | — | — | 35.1 | — | — | — | |
| TIPSBackbone=Qwen2.5-7B Instruct2026.05 | — | — | 50.3 | — | — | — | |
| ZeroSearchBackbone=Qwen2.5-3B Instruct2026.05 | — | — | 28.8 | — | — | — | |
| ZeroSearchBackbone=Qwen2.5-7B Instruct2026.05 | — | — | 37 | — | — | — |