Multi-hop Question Answering on MuSiQue (test)
55.68F1SEARCH-R
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| SEARCH-RBackbone=gpt-4o-mini2026.04 | 55.68 | — | — | — | — | — | 45 | — | — | |
| MultiCube-RAGGenerator=GPT-4o-mini2026.02 | 50.9 | — | — | — | — | — | 39.5 | — | — | |
| MultiCube-RAGGenerator=Llama3.3-70B-Instruct2026.02 | 49.7 | — | — | — | — | — | 38.8 | — | — | |
| ProGraph-R1Base Model=Qwen2.5-7B-Instruct, Optimization Method=Training, Knowledge Interaction=Graph-based knowledge2026.01 | 49.47 | — | — | — | — | — | 3,984 | — | — | |
| HippoRAG 2Generator=GPT-4o-mini2026.02 | 49.3 | — | — | — | — | — | 35 | — | — | |
| HippoRAG 2Generator=Llama3.3-70B-Instruct2026.02 | 48.6 | — | — | — | — | — | 37.2 | — | — | |
| HippoRAG2Backbone=Llama-3.3-70B-Instruct2026.02 | 48.6 | — | — | — | — | — | — | — | — | |
| HELPBackbone=Llama-3.3-70B-Instruct2026.02 | 48.4 | — | — | — | — | — | — | — | — | |
| ProGraph-R1Base Model=Qwen2.5-3B-Instruct, Optimization Method=Training, Knowledge Interaction=Graph-based knowledge2026.01 | 47.27 | — | — | — | — | — | 3,750 | — | — | |
| Graph-R1Base Model=Qwen2.5-7B-Instruct, Optimization Method=Training, Knowledge Interaction=Graph-based knowledge2026.01 | 46.17 | — | — | — | — | — | 3,672 | — | — | |
| NV-Embed-v2 (7B)Generator=GPT-4o-mini2026.02 | 46 | — | — | — | — | — | 32.8 | — | — | |
| NV-Embed-v2 (7B)Generator=Llama3.3-70B-Instruct2026.02 | 45.7 | — | — | — | — | — | 34.7 | — | — | |
| NV-Embed-v2Backbone=Llama-3.3-70B-Instruct2026.02 | 45.7 | — | — | — | — | — | — | — | — | |
| CatRAGQA reader=Llama-3.3-70B-Instruct2026.02 | 45 | — | — | — | — | — | — | — | — | |
| IGMiRAG2026.02 | 45 | — | — | — | — | — | 33 | — | — | |
| GritLM-7BBackbone=Llama-3.3-70B-Instruct2026.02 | 44.8 | — | — | — | — | — | — | — | — | |
| HippoRAG 2QA reader=Llama-3.3-70B-Instruct, Retrieval=Structure-Aware RAG2026.02 | 43.2 | — | — | — | — | — | — | — | — | |
| LightRAGQA reader=Llama-3.3-70B-Instruct, Retrieval=Structure-Aware RAG2026.02 | 43 | — | — | — | — | — | — | — | — | |
| GraphRAGGenerator=GPT-4o-mini2026.02 | 42 | — | — | — | — | — | 27 | — | — | |
| HyperGraphRAGBackbone=Llama-3.3-70B-Instruct, Local Reproduction=true2026.02 | 41.6 | — | — | — | — | — | — | — | — | |
| GTE-Qwen2-7BBackbone=Llama-3.3-70B-Instruct2026.02 | 40.9 | — | — | — | — | — | — | — | — | |
| Graph-R1Base Model=Qwen2.5-3B-Instruct, Optimization Method=Training, Knowledge Interaction=Graph-based knowledge2026.01 | 40.51 | — | — | — | — | — | 3,281 | — | — | |
| Naive RAGTop-k retrieved slices=52026.02 | 40.3 | — | — | — | — | — | 28.6 | — | — | |
| NodeRAG2026.02 | 40.1 | — | — | — | — | — | 27.8 | — | — | |
| RAPTORGenerator=GPT-4o-mini2026.02 | 39.2 | — | — | — | — | — | 27.7 | — | — | |
| GraphRAGGenerator=Llama3.3-70B-Instruct2026.02 | 38.5 | — | — | — | — | — | 27.3 | — | — | |
| GraphRAGBackbone=Llama-3.3-70B-Instruct2026.02 | 38.5 | — | — | — | — | — | — | — | — | |
| Naive RAGTop-k retrieved slices=32026.02 | 38.4 | — | — | — | — | — | 27.4 | — | — | |
| text-embedding-3-smallQA reader=Llama-3.3-70B-Instruct, Retrieval=Standard Retrieval2026.02 | 36.1 | — | — | — | — | — | — | — | — | |
| RAPTORQA reader=Llama-3.3-70B-Instruct, Retrieval=Structure-Aware RAG2026.02 | 36 | — | — | — | — | — | — | — | — | |
| HippoRAGGenerator=GPT-4o-mini2026.02 | 35.9 | — | — | — | — | — | 24 | — | — | |
| RankCoTGenerator=GPT-4o-mini2026.02 | 35.8 | — | — | — | — | — | 23.8 | — | — | |
| HippoRAGGenerator=Llama3.3-70B-Instruct2026.02 | 35.1 | — | — | — | — | — | 26.2 | — | — | |
| HippoRAGBackbone=Llama-3.3-70B-Instruct2026.02 | 35.1 | — | — | — | — | — | — | — | — | |
| GTR (T5-base)QA reader=Llama-3.3-70B-Instruct, Retrieval=Standard Retrieval2026.02 | 34.6 | — | — | — | — | — | — | — | — | |
| GTRBackbone=Llama-3.3-70B-Instruct2026.02 | 34.6 | — | — | — | — | — | — | — | — | |
| RoMemMemory category=Structured memory baselines2026.06 | 34.2 | — | — | — | 24.2 | — | 16.1 | — | 37.1 | |
| IQATRLLM Backbone=Llama-3.1-8B, Retriever=Contriever, Retriever Fine-tuning=ReSCORE2025.05 | 32.7 | — | — | — | — | — | 23.4 | — | — | |
| Hyper-RAG2026.02 | 32.6 | — | — | — | — | — | 22.6 | — | — | |
| LightRAG2026.02 | 32.4 | — | — | — | — | — | 22.6 | — | — | |
| IRCOTLLM Backbone=Flan-T5-XL, Retriever=BM252025.05 | 31.8 | — | — | — | — | — | 22 | — | — | |
| Adaptive-RAGLLM Backbone=Flan-T5-XL, Retriever=BM252025.05 | 31.8 | — | — | — | — | — | 23.6 | — | — | |
| Naive RAGTop-k retrieved slices=12026.02 | 31.4 | — | — | — | — | — | 21.4 | — | — | |
| DEC (Ours)Backbone=GPT-4o, Retrieval=with Retrieval2025.06 | 31.3 | 3.22 | 27.91 | 30.72 | — | — | — | — | — | |
| ContrieverQA reader=Llama-3.3-70B-Instruct, Retrieval=Standard Retrieval2026.02 | 31.3 | — | — | — | — | — | — | — | — | |
| ContrieverBackbone=Llama-3.3-70B-Instruct2026.02 | 31.3 | — | — | — | — | — | — | — | — | |
| PathRAG2026.02 | 31.1 | — | — | — | — | — | 19.3 | — | — | |
| LinearRAGBackbone=Llama-3.3-70B-Instruct, Local Reproduction=true2026.02 | 31 | — | — | — | — | — | — | — | — | |
| GRPOBackbone=Qwen3-14B2026.06 | 29.9 | — | — | — | 31.6 | — | 19.4 | — | — | |
| OThink-SRR1Base model=Qwen2.5-7B-Instruct2026.03 | 29.85 | — | — | — | — | — | 20.73 | — | — | |
| NativeRAGBackbone=gpt-4o-mini2026.04 | 29.82 | — | — | — | — | — | 19 | — | — | |
| RAPTORGenerator=Llama3.3-70B-Instruct2026.02 | 28.9 | — | — | — | — | — | 20.7 | — | — | |
| RAPTORBackbone=Llama-3.3-70B-Instruct2026.02 | 28.9 | — | — | — | — | — | — | — | — | |
| BM25Generator=Llama3.3-70B-Instruct2026.02 | 28.8 | — | — | — | — | — | 20.3 | — | — | |
| BM25Backbone=Llama-3.3-70B-Instruct2026.02 | 28.8 | — | — | — | — | — | — | — | — | |
| ARBORBackbone=Qwen3-14B2026.06 | 28.7 | — | — | — | 31.6 | — | 19.1 | — | — | |
| ReSearchBase model=Qwen2.5-7B-Instruct2026.03 | 28.66 | — | — | — | — | — | 19.07 | — | — | |
| Direct CoTBackbone=GPT-4o, Retrieval=without Retrieval2025.06 | 28.59 | 1 | 25.5 | 29.5 | — | — | — | — | — | |
| IRCoTGenerator=GPT-4o-mini2026.02 | 27.8 | — | — | — | — | — | 18.4 | — | — | |
| SearChainBackbone=GPT-4o, Retrieval=with Retrieval2025.06 | 27.77 | 2.62 | 23.45 | 27.66 | — | — | — | — | — | |
| GenGroundRetrieval Strategy=w/ Retrieval, Backbone=gpt-3.5-turbo2024.06 | 27.36 | — | — | — | 20.24 | 24.77 | — | — | — | |
| BM25Generator=GPT-4o-mini2026.02 | 27.3 | — | — | — | — | — | 17.9 | — | — | |
| DAPOBackbone=Qwen3-14B2026.06 | 27.1 | — | — | — | 27.3 | — | 17.2 | — | — | |
| GRPOBackbone=Qwen3-4B2026.06 | 26.9 | — | — | — | 26.5 | — | 17 | — | — | |
| ARBORBackbone=Qwen3-8B2026.06 | 26.8 | — | — | — | 29.5 | — | 18.3 | — | — | |
| GRPOBackbone=Qwen3-8B2026.06 | 26.7 | — | — | — | 26.6 | — | 17.4 | — | — | |
| Cog-RAG2026.02 | 26.6 | — | — | — | — | — | 15.5 | — | — | |
| Self-AskBackbone=GPT-4o, Retrieval=with Retrieval2025.06 | 26.17 | 3.44 | 20.2 | 22.8 | — | — | — | — | — | |
| NoneQA reader=Llama-3.3-70B-Instruct, Retrieval=None2026.02 | 26.1 | — | — | — | — | — | — | — | — | |
| NoneBackbone=Llama-3.3-70B-Instruct2026.02 | 26.1 | — | — | — | — | — | — | — | — | |
| DAPOBackbone=Qwen3-4B2026.06 | 25.9 | — | — | — | 25.2 | — | 16.4 | — | — | |
| DAPOBackbone=Qwen3-8B2026.06 | 25.8 | — | — | — | 26.2 | — | 17 | — | — | |
| ARBORBackbone=Qwen3-4B2026.06 | 25.7 | — | — | — | 27.8 | — | 17 | — | — | |
| GRG w/ decompositionRetrieval Strategy=w/ Retrieval, Backbone=gpt-3.5-turbo, Decomposition=true2024.06 | 24.87 | — | — | — | 17.91 | 22.33 | — | — | — | |
| Direct RAGBackbone=GPT-4o, Retrieval=with Retrieval2025.06 | 24.7 | 1 | 22 | 27 | — | — | — | — | — | |
| GenGroundBackbone=GPT-4o, Retrieval=with Retrieval2025.06 | 24.63 | 5 | 23 | 22.5 | — | — | — | — | — | |
| Search-R1Base model=Qwen2.5-7B-Instruct2026.03 | 24.58 | — | — | — | — | — | 16.88 | — | — | |
| Adaptive-NoteLLM Backbone=GPT-3.5, Retriever=BM252025.05 | 24.2 | — | — | — | — | — | 13.2 | — | — | |
| OThink-SRR1Base model=Qwen2.5-3B-Instruct2026.03 | 23.94 | — | — | — | — | — | 14.98 | — | — | |
| Llama-3.1-8B BaselineLLM Backbone=Llama-3.1-8B, Retriever=Contriever2025.05 | 23.8 | — | — | — | — | — | 15.2 | — | — | |
| Llama-3.1-8B BaselineLLM Backbone=Llama-3.1-8B, Retriever=BM252025.05 | 23.6 | — | — | — | — | — | 15.2 | — | — | |
| CoTRetrieval Strategy=w/o Retrieval, Backbone=gpt-3.5-turbo2024.06 | 23.35 | — | — | — | 13.21 | 17.85 | — | — | — | |
| BM25QA reader=Llama-3.3-70B-Instruct, Retrieval=Standard Retrieval2026.02 | 22.9 | — | — | — | — | — | — | — | — | |
| Search-R1Base Model=Qwen2.5-7B-Instruct, Optimization Method=Training, Knowledge Interaction=Chunk-based knowledge2026.01 | 22.35 | — | — | — | — | — | 1,484 | — | — | |
| GPT-4o-mini2026.02 | 21.7 | — | — | — | — | — | 12.4 | — | — | |
| Step-AccumulateMemory category=Raw-buffer memory baselines2026.06 | 21.5 | — | — | — | 16.1 | — | 12.9 | — | 21 | |
| GraphRAGBackbone=gpt-4o-mini2026.04 | 21.4 | — | — | — | — | — | 6 | — | — | |
| RetGenRetrieval Strategy=w/ Retrieval, Backbone=gpt-3.5-turbo2024.06 | 21.04 | — | — | — | 17.69 | 20.19 | — | — | — | |
| DEC (Ours)Backbone=Llama-3.1-8B-Instruct, Retrieval=with Retrieval2025.06 | 20.96 | 3.43 | 17.21 | 19.26 | — | — | — | — | — | |
| MementoMemory category=Structured memory baselines2026.06 | 20.8 | — | — | — | 14.5 | — | 12.9 | — | 24.2 | |
| GenRead w/ decompositionRetrieval Strategy=w/o Retrieval, Backbone=gpt-3.5-turbo, Decomposition=true2024.06 | 20.13 | — | — | — | 17.58 | 20.62 | — | — | — | |
| DSPyRetrieval Strategy=w/ Retrieval, Backbone=gpt-3.5-turbo2024.06 | 20.11 | — | — | — | 13.4 | 17.4 | — | — | — | |
| ReACTLLM Backbone=GPT-3.5, Retriever=BM252025.05 | 19.7 | — | — | — | — | — | 10.2 | — | — | |
| ReSearchBase model=Qwen2.5-3B-Instruct2026.03 | 19.32 | — | — | — | — | — | 12.45 | — | — | |
| Self-RAGLLM Backbone=GPT-3.5, Retriever=BM252025.05 | 19.2 | — | — | — | — | — | 10.6 | — | — | |
| FLARELLM Backbone=GPT-3.5, Retriever=BM252025.05 | 18.7 | — | — | — | — | — | 11.2 | — | — | |
| SearChainBackbone=Llama-3.1-8B-Instruct, Retrieval=with Retrieval2025.06 | 17.77 | 3.73 | 13.05 | 17.47 | — | — | — | — | — | |
| Mem0Memory category=Structured memory baselines2026.06 | 17.4 | — | — | — | 11.3 | — | 11.3 | — | 17.7 | |
| Least-to-MostMemory category=Memory-free baselines2026.06 | 17.3 | — | — | — | 8.1 | — | 6.5 | — | 21 | |
| Vanilla ChatBackbone=GPT-4o, Retrieval=without Retrieval2025.06 | 17.28 | 1 | 10.5 | 12.5 | — | — | — | — | — |