Multi-hop Question Answering on Average (MuSiQue, 2WikiMQA, HotpotQA)
43.3EMGraph + LLM
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Graph + LLMKG Source=D-SynthKG-8b2024.10 | 43.3 | 56.9 | — | — | — | — | |
| Graph + LLMKG Source=Llama-3-70b2024.10 | 41.3 | 54.8 | — | — | — | — | |
| HippoRAGKG Source=GPT-4o2024.10 | 40.3 | 54.6 | — | — | — | — | |
| Dense + LLMKG Source=None2024.10 | 39.4 | 53.6 | — | — | — | — | |
| GraphRAG (local)KG Source=GPT-4o2024.10 | 39 | 49.1 | — | — | — | — | |
| GraphRAG (drift)KG Source=GPT-4o2024.10 | 38.4 | 51.3 | — | — | — | — | |
| R1-SearcherBackbone=7B-Base, Train mix=HQA+2Wiki2026.05 | 37.1 | — | — | 44.2 | 51.3 | 15.8 | |
| LlamaIndexKG Source=D-SynthKG-8b2024.10 | 36.7 | 48.5 | — | — | — | — | |
| Dense RetrieverKG Source=None2024.10 | 36.3 | 50.5 | — | — | — | — | |
| Chain-of-TripletKG Source=D-SynthKG-8b2024.10 | 35.4 | 49 | — | — | — | — | |
| Typed+Shape (ours)Backbone=7B-Inst, Train mix=NQ+HQA2026.05 | 35.4 | — | — | 39.1 | 41.5 | 26.7 | |
| LlamaIndexKG Source=Llama-3-70b2024.10 | 34.7 | 45.9 | — | — | — | — | |
| Typed BaseBackbone=7B-Inst, Train mix=NQ+HQA2026.05 | 34.6 | — | — | 37.8 | 40.5 | 25.5 | |
| GlobalRAGBackbone=7B-Inst, Train mix=2Wiki+HQA2026.05 | 33.6 | — | — | 38.2 | 47.8 | 14.9 | |
| Binary GapBackbone=7B-Inst, Train mix=NQ+HQA2026.05 | 33.4 | — | — | 36.6 | 39 | 24.6 | |
| Search-R1 (PPO)Backbone=7B-Inst, Train mix=NQ+HQA2026.05 | 31 | — | — | 37 | 41.4 | 14.6 | |
| LlamaIndexKG Source=Llama-3-8b2024.10 | 30.8 | 42.5 | — | — | — | — | |
| Chain-of-TripletKG Source=Llama-3-70b2024.10 | 30.3 | 41.5 | — | — | — | — | |
| Search-R1 (GRPO)Backbone=7B-Inst, Train mix=NQ+HQA2026.05 | 29.8 | — | — | 38.6 | 34.6 | 16.2 | |
| Search-OnlyBackbone=7B-Inst, Train mix=NQ+HQA2026.05 | 29.8 | — | — | 38.6 | 34.6 | 16.2 | |
| StepSearchBackbone=7B-Inst, Train mix=MuSiQue2026.05 | 28.2 | — | — | 34.8 | 36.3 | 13.4 | |
| Graph + LLMKG Source=Llama-3-8b2024.10 | 28.1 | 40.2 | — | — | — | — | |
| Chain-of-TripletKG Source=Llama-3-8b2024.10 | 23.8 | 36.5 | — | — | — | — | |
| Naive RAGBackbone=7B-Inst2026.05 | 19.7 | — | — | 29.9 | 23.5 | 5.8 | |
| NoneKG Source=None, Retrieval=None2024.10 | 19.3 | 33.3 | — | — | — | — | |
| SFTBackbone=7B-Inst, Train mix=NQ+HQA2026.05 | 18.1 | — | — | 21.7 | 25.9 | 6.6 | |
| IRCoTBackbone=7B-Inst2026.05 | 11.8 | — | — | 13.3 | 14.9 | 7.2 | |
| BubbleRAGModel Scale=30B & 8B2026.03 | — | 63.02 | 66.63 | — | — | — | |
| Clue-RAGModel Scale=30B & 8B2026.03 | — | 43.29 | 48.65 | — | — | — | |
| CoT + LLMModel Scale=30B & 8B2026.03 | — | 14.51 | 23.35 | — | — | — | |
| HippoRAG2Model Scale=30B & 8B2026.03 | — | 60.5 | 64.4 | — | — | — | |
| LightRAG (global)Model Scale=30B & 8B2026.03 | — | 25.56 | 27.17 | — | — | — | |
| LightRAG (hybrid)Model Scale=30B & 8B2026.03 | — | 47.74 | 49.92 | — | — | — | |
| LightRAG (local)Model Scale=30B & 8B2026.03 | — | 50.73 | 52.94 | — | — | — | |
| Naive RAGModel Scale=30B & 8B2026.03 | — | 53.51 | 56.71 | — | — | — | |
| RAPTORModel Scale=30B & 8B2026.03 | — | 44.7 | 46.97 | — | — | — | |
| ToGModel Scale=30B & 8B2026.03 | — | 24.15 | 30.16 | — | — | — | |
| Vanilla LLMModel Scale=30B & 8B2026.03 | — | 12.97 | 23.88 | — | — | — |