Multi-Hop Question Answering on HotpotQA (SubEM, F1)
58.9F1GraphRAG-Router
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GraphRAG-RouterSystem=Our Method2026.03 | 58.9 | — | |
| HippoRAG2System=RAG-Based2026.03 | 50.1 | — | |
| LinearRAGSystem=RAG-Based2026.03 | 48.7 | — | |
| Graph-R1System=RL-based Agent2026.03 | 46.8 | — | |
| Router-R1System=RL-based Agent2026.03 | 46.7 | — | |
| Vanilla RAGSystem=RAG-Based2026.03 | 44.5 | — | |
| Search-o1System=Training-free Agent2026.03 | 43.9 | — | |
| CoTSystem=Basic LLM2026.03 | 43.6 | — | |
| Stable-RAGBackbone=Qwen3-8B, Retriever=Contriever, exhaustive decoding=true2026.01 | 42.94 | 36.89 | |
| GraphSearchSystem=Training-free Agent2026.03 | 42.5 | — | |
| Stable-RAGBackbone=Qwen3-8B, Retriever=Contriever, exhaustive decoding=false2026.01 | 41.78 | 35.73 | |
| RAPTORSystem=RAG-Based2026.03 | 40.9 | — | |
| ATMBackbone=Qwen3-8B, Retriever=Contriever2026.01 | 40.69 | 35.12 | |
| Stable-RAGBackbone=LLaMA3-8B-Instruct, Retriever=Contriever, exhaustive decoding=true2026.01 | 40.16 | 39.12 | |
| Stable-RAGBackbone=LLaMA3-8B-Instruct, Retriever=Contriever, exhaustive decoding=false2026.01 | 39.87 | 38.91 | |
| RetRobustBackbone=Qwen3-8B, Retriever=Contriever2026.01 | 39.39 | 32.77 | |
| Pos2DistillBackbone=Qwen3-8B, Retriever=Contriever2026.01 | 39.11 | 33.72 | |
| RAATBackbone=Qwen3-8B, Retriever=Contriever2026.01 | 39.06 | 33.54 | |
| GraphRAGSystem=RAG-Based2026.03 | 38.8 | — | |
| Vanilla RAGBackbone=Qwen3-8B, Retriever=Contriever2026.01 | 38.66 | 33.14 | |
| Ms-PoEBackbone=Qwen3-8B, Retriever=Contriever2026.01 | 38.21 | 32.98 | |
| HyperGraphRAGSystem=RAG-Based2026.03 | 37.6 | — | |
| Search-R1System=RL-based Agent2026.03 | 37.1 | — | |
| ATMBackbone=LLaMA3-8B-Instruct, Retriever=Contriever2026.01 | 36.97 | 34.36 | |
| RAATBackbone=LLaMA3-8B-Instruct, Retriever=Contriever2026.01 | 36.12 | 33.58 | |
| Direct InferSystem=Basic LLM2026.03 | 36.1 | — | |
| Pos2DistillBackbone=LLaMA3-8B-Instruct, Retriever=Contriever2026.01 | 35.79 | 32.73 | |
| Stable-RAGBackbone=Qwen3-8B, Retriever=DPR, exhaustive decoding=true2026.01 | 35.78 | 31.77 | |
| RetRobustBackbone=LLaMA3-8B-Instruct, Retriever=Contriever2026.01 | 35.34 | 31.46 | |
| Vanilla SFTBackbone=Qwen3-8B, Retriever=Contriever2026.01 | 34.15 | 28.36 | |
| Vanilla RAGBackbone=LLaMA3-8B-Instruct, Retriever=Contriever2026.01 | 34.08 | 30.73 | |
| RAATBackbone=Qwen3-8B, Retriever=DPR2026.01 | 33.75 | 27.21 | |
| Ms-PoEBackbone=LLaMA3-8B-Instruct, Retriever=Contriever2026.01 | 33.65 | 30.17 | |
| ATMBackbone=Qwen3-8B, Retriever=DPR2026.01 | 33.43 | 29.07 | |
| Stable-RAGBackbone=Qwen3-8B, Retriever=DPR, exhaustive decoding=false2026.01 | 33.26 | 30.15 | |
| RetRobustBackbone=Qwen3-8B, Retriever=DPR2026.01 | 33.06 | 26.83 | |
| Stable-RAGBackbone=LLaMA3-8B-Instruct, Retriever=DPR, exhaustive decoding=true2026.01 | 32.12 | 30.41 | |
| Pos2DistillBackbone=Qwen3-8B, Retriever=DPR2026.01 | 31.88 | 26.53 | |
| Stable-RAGBackbone=LLaMA3-8B-Instruct, Retriever=DPR, exhaustive decoding=false2026.01 | 31.68 | 29.48 | |
| Vanilla SFTBackbone=LLaMA3-8B-Instruct, Retriever=Contriever2026.01 | 31.58 | 27.25 | |
| Vanilla RAGBackbone=Qwen3-8B, Retriever=DPR2026.01 | 31.33 | 26.17 | |
| Ms-PoEBackbone=Qwen3-8B, Retriever=DPR2026.01 | 31.02 | 25.93 | |
| RetRobustBackbone=LLaMA3-8B-Instruct, Retriever=DPR2026.01 | 30.36 | 26.96 | |
| Vanilla SFTBackbone=LLaMA3-8B-Instruct, Retriever=DPR2026.01 | 29.85 | 24.63 | |
| Vanilla SFTBackbone=Qwen3-8B, Retriever=DPR2026.01 | 29.77 | 25.35 | |
| ATMBackbone=LLaMA3-8B-Instruct, Retriever=DPR2026.01 | 29.31 | 28.55 | |
| Pos2DistillBackbone=LLaMA3-8B-Instruct, Retriever=DPR2026.01 | 28.91 | 26.45 | |
| RAATBackbone=LLaMA3-8B-Instruct, Retriever=DPR2026.01 | 28.79 | 26.35 | |
| Ms-PoEBackbone=LLaMA3-8B-Instruct, Retriever=DPR2026.01 | 28.57 | 26.12 | |
| Vanilla RAGBackbone=LLaMA3-8B-Instruct, Retriever=DPR2026.01 | 28.22 | 25.66 | |
| Naive GenerationBackbone=Qwen3-8B, Retriever=Contriever2026.01 | 24.86 | 19.54 | |
| Naive GenerationBackbone=Qwen3-8B, Retriever=DPR2026.01 | 24.86 | 19.54 | |
| Direct GenerationBackbone=LLaMA3-8B-Instruct, Retriever=Contriever2026.01 | 22.87 | 21.39 | |
| Direct GenerationBackbone=LLaMA3-8B-Instruct, Retriever=DPR2026.01 | 22.87 | 21.39 |