Multi-hop Question Answering on HotpotQA (EM, LLM Judge)
80LLM Judge ScoreINSES + Router
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| INSES + RouterCategory=Ours2026.03 | 80 | 68 | — | |
| Naïve RAGCategory=Text-based, Top-k=102026.03 | 77 | 67 | — | |
| LightRAGCategory=Graph-based2026.03 | 77 | 67 | — | |
| SiReRAGCategory=Graph-based2026.03 | 75 | 61 | — | |
| MemGraphRAGRetrieval Strategy=Graph-based Retrieval-Augmented-Generation2026.05 | 71.6 | — | 67.2 | |
| Naïve RAGCategory=Text-based, Top-k=52026.03 | 71 | 62 | — | |
| ConRAGLLM backbone=Gemma-4-31B, Type=Ours2026.05 | 70.4 | — | 68.4 | |
| RAPTORCategory=Graph-based2026.03 | 70 | 68 | — | |
| Youtu-GraphRAGLLM backbone=Gemma-4-31B, Type=Graph-based RAG2026.05 | 69.1 | — | 67.4 | |
| GFM-RAGRetrieval Strategy=Graph-based Retrieval-Augmented-Generation2026.05 | 67.7 | — | 64.1 | |
| LinearRAGRetrieval Strategy=Graph-based Retrieval-Augmented-Generation2026.05 | 67.3 | — | 65.3 | |
| HippoRAG2Retrieval Strategy=Graph-based Retrieval-Augmented-Generation2026.05 | 67.2 | — | 65.2 | |
| M2RBackbone=Qwen2.5-7B-Instruct2026.04 | 65.98 | 44.11 | — | |
| E^2GraphRAGRetrieval Strategy=Graph-based Retrieval-Augmented-Generation2026.05 | 65.7 | — | 63.1 | |
| LogicRAGRetrieval Strategy=Graph-based Retrieval-Augmented-Generation2026.05 | 65.6 | — | 55.8 | |
| ReSearchBackbone=Qwen2.5-7B-Instruct2026.04 | 63.62 | 43.52 | — | |
| ConRAGLLM backbone=GPT-4o-mini, Type=Ours2026.05 | 63.5 | — | 63 | |
| HyDECategory=Text-based2026.03 | 63 | 57 | — | |
| GraphRAGCategory=Graph-based, Top-k=52026.03 | 63 | 43 | — | |
| LogicRAGLLM backbone=GPT-4o-mini, Type=Reasoning-based RAG2026.05 | 62.6 | — | 54.8 | |
| LogicRAGLLM backbone=Gemma-4-31B, Type=Reasoning-based RAG2026.05 | 62.4 | — | 60.7 | |
| KGPRetrieval Strategy=Graph-based Retrieval-Augmented-Generation2026.05 | 62.1 | — | 62.7 | |
| LightRAGRetrieval Strategy=Graph-based Retrieval-Augmented-Generation2026.05 | 62 | — | 61.4 | |
| HippoRAG2LLM backbone=GPT-4o-mini, Type=Graph-based RAG2026.05 | 61.9 | — | 56.7 | |
| COFTBackbone=Qwen2.5-7B-Instruct2026.04 | 61.71 | 41.08 | — | |
| HippoRAGRetrieval Strategy=Graph-based Retrieval-Augmented-Generation2026.05 | 61.4 | — | 58.4 | |
| GraphRAGCategory=Graph-based, Top-k=102026.03 | 61 | 47 | — | |
| RAPTORRetrieval Strategy=Graph-based Retrieval-Augmented-Generation2026.05 | 61 | — | 57 | |
| RetrievalRetrieval Strategy=Vanilla Retrieval-Augmented-Generation, Top-k Retrieval=52026.05 | 60.3 | — | 58.5 | |
| SUREBackbone=Qwen2.5-7B-Instruct2026.04 | 60.16 | 39.56 | — | |
| RetrievalRetrieval Strategy=Vanilla Retrieval-Augmented-Generation, Top-k Retrieval=32026.05 | 58.9 | — | 55.4 | |
| RAPTORLLM backbone=GPT-4o-mini, Type=Graph-based RAG2026.05 | 57.8 | — | 48.1 | |
| LightRAGLLM backbone=GPT-4o-mini, Type=Graph-based RAG2026.05 | 57.7 | — | 47.8 | |
| KGPLLM backbone=GPT-4o-mini, Type=Graph-based RAG2026.05 | 57.1 | — | 46.4 | |
| GLM-4Category=LLM only, Protocol=Few-shot CoT2026.03 | 57 | 51 | — | |
| M2RBackbone=Qwen2.5-3B-Instruct2026.04 | 56.46 | 38.7 | — | |
| HippoRAG2LLM backbone=Gemma-4-31B, Type=Graph-based RAG2026.05 | 55.9 | — | 54.1 | |
| ReSearchBackbone=Qwen2.5-3B-Instruct2026.04 | 55.7 | 38.78 | — | |
| HippoRAGLLM backbone=GPT-4o-mini, Type=Graph-based RAG2026.05 | 55.6 | — | 53.7 | |
| HippoRAGLLM backbone=Gemma-4-31B, Type=Graph-based RAG2026.05 | 55.3 | — | 54.1 | |
| Vanilla RAG (Top-5)LLM backbone=GPT-4o-mini, Type=Text RAG2026.05 | 53.9 | — | 44.1 | |
| Vanilla RAG (Top-3)LLM backbone=GPT-4o-mini, Type=Text RAG2026.05 | 53.1 | — | 43.2 | |
| Iter-RetGenBackbone=Qwen2.5-7B-Instruct2026.04 | 52.22 | 34.36 | — | |
| IRCoTBackbone=Qwen2.5-7B-Instruct2026.04 | 52.06 | 30.33 | — | |
| Vanilla RAG (Top-5)LLM backbone=Gemma-4-31B, Type=Text RAG2026.05 | 52 | — | 51.1 | |
| SUREBackbone=Qwen2.5-3B-Instruct2026.04 | 51.23 | 35.44 | — | |
| COFTBackbone=Qwen2.5-3B-Instruct2026.04 | 50.88 | 36.17 | — | |
| RetrievalRetrieval Strategy=Vanilla Retrieval-Augmented-Generation, Top-k Retrieval=12026.05 | 50.4 | — | 48.8 | |
| Naive RAGBackbone=Qwen2.5-7B-Instruct2026.04 | 49.59 | 31.9 | — | |
| LinearRAGLLM backbone=GPT-4o-mini, Type=Graph-based RAG2026.05 | 49.1 | — | 50.1 | |
| GLM-4Category=LLM only, Protocol=Direct2026.03 | 49 | 41 | — | |
| Vanilla RAG (Top-1)LLM backbone=GPT-4o-mini, Type=Text RAG2026.05 | 48.6 | — | 38.4 | |
| Vanilla RAG (Top-3)LLM backbone=Gemma-4-31B, Type=Text RAG2026.05 | 48.3 | — | 47.6 | |
| IRCoTCategory=Text-based2026.03 | 48 | 37 | — | |
| LinearRAGLLM backbone=Gemma-4-31B, Type=Graph-based RAG2026.05 | 46.1 | — | 45.7 | |
| GraphRAGLLM backbone=GPT-4o-mini, Type=Graph-based RAG2026.05 | 45.2 | — | 39.6 | |
| LazyGraphRAGRetrieval Strategy=Graph-based Retrieval-Augmented-Generation2026.05 | 43.8 | — | 52.7 | |
| MS-GraphRAGRetrieval Strategy=Graph-based Retrieval-Augmented-Generation2026.05 | 43.5 | — | 51.6 | |
| GPT-3.5-TurboRetrieval Strategy=Direct Zero-shot LLM Inference2026.05 | 42.5 | — | 32.7 | |
| Iter-RetGenBackbone=Qwen2.5-3B-Instruct2026.04 | 42.22 | 24.63 | — | |
| G-retrieverRetrieval Strategy=Graph-based Retrieval-Augmented-Generation2026.05 | 41.8 | — | 44 | |
| G-RetrieverLLM backbone=GPT-4o-mini, Type=Graph-based RAG2026.05 | 40.9 | — | 28.5 | |
| IRCoTBackbone=Qwen2.5-3B-Instruct2026.04 | 40.6 | 23.63 | — | |
| GPT-4o-miniRetrieval Strategy=Direct Zero-shot LLM Inference2026.05 | 39.7 | — | 38.1 | |
| Vanilla RAG (Top-1)LLM backbone=Gemma-4-31B, Type=Text RAG2026.05 | 38.1 | — | 39.2 | |
| Naive RAGBackbone=Qwen2.5-3B-Instruct2026.04 | 37.23 | 21.04 | — | |
| Zero-shotLLM backbone=GPT-4o-mini, Type=Direct LLM2026.05 | 36.3 | — | 38.7 | |
| Naive GenerationBackbone=Qwen2.5-7B-Instruct2026.04 | 30.64 | 19.18 | — | |
| Zero-shotLLM backbone=Gemma-4-31B, Type=Direct LLM2026.05 | 30.6 | — | 31.5 | |
| Llama3-8BRetrieval Strategy=Direct Zero-shot LLM Inference2026.05 | 28.2 | — | 30.8 | |
| Naive GenerationBackbone=Qwen2.5-3B-Instruct2026.04 | 18.45 | 12.05 | — | |
| Llama3-13BRetrieval Strategy=Direct Zero-shot LLM Inference2026.05 | 17 | — | 24.9 | |
| AdaptiveRAGTraining Paradigm=Training-free, Backbone=Qwen2.5-7B2026.06 | — | 37.1 | — | |
| AR W/ RET@1Training Paradigm=Training-free, Backbone=Qwen2.5-7B, Retrieval Strategy=RET@12026.06 | — | 47.4 | — | |
| AR W/ RET@10Training Paradigm=Training-free, Backbone=Qwen2.5-7B, Retrieval Strategy=RET@102026.06 | — | 45.1 | — | |
| AR W/ RET@ADAPTIVETraining Paradigm=Training-free, Backbone=Qwen2.5-7B, Retrieval Strategy=ADAPTIVE2026.06 | — | 37.3 | — | |
| AR W/ RET@STATICTraining Paradigm=Training-free, Backbone=Qwen2.5-7B, Retrieval Strategy=STATIC2026.06 | — | 40.3 | — | |
| DLM W/ RET@STATICTraining Paradigm=Training-free, Backbone=DREAM-7B, Retrieval Strategy=STATIC, Diffusion Threshold (tau_c)=0.92026.06 | — | 39.9 | — | |
| ReActTraining Paradigm=Training-free, Backbone=Qwen2.5-7B2026.06 | — | 40.1 | — | |
| SARDITraining Paradigm=Training-free, Backbone=DREAM-7B, Retrieval Strategy=SARDI, Diffusion Threshold (tau_c)=0.92026.06 | — | 48.5 | — | |
| SARDITraining Paradigm=Training-free, Backbone=DREAM-7B, Retrieval Strategy=SARDI, Diffusion Threshold (tau_c)=0.952026.06 | — | 48.7 | — | |
| Search-R1Training Paradigm=RL-trained, Backbone=Qwen2.5-7B2026.06 | — | 50.3 | — |