Multi-hop Question Answering on 2WikiMultiHopQA
82.1EMSynPlanResearch-R1
Evaluation results
| Method | Links | ||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SynPlanResearch-R1Model Backbone=Qwen3-8B2026.03 | 82.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SynPlanResearch-R1Model Backbone=Qwen3-4B2026.03 | 79.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Gfm-RetrieverEvaluation Protocol=+ IRCoT2026.03 | 78.8 | 79.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | 82.3 | 79.4 | — | — | — | — | — | |
| SimpleDeepSearcherModel Backbone=Qwen3-8B2026.03 | 77.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| HippoRAG 2Evaluation Protocol=Standalone2026.03 | 75.4 | 78.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | 80.3 | 78.7 | — | — | — | — | — | |
| HippoRAG 2Avg. Rank=4.72026.05 | 75.4 | 78.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Gfm-RetrieverEvaluation Protocol=Standalone2026.03 | 74.3 | 78.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | 80.4 | 79.1 | — | — | — | — | — | |
| SimpleDeepSearcherModel Backbone=Qwen3-4B2026.03 | 74.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SAGEvariant=+2 round, Avg. Rank=2.52026.05 | 74 | 80.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| STRIDE-FTBackbone=Qwen3-8B, Fine-tuning=true2026.04 | 73.8 | 79.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | 80.6 | 79.8 | — | — | — | — | — | |
| PropRAGEvaluation Protocol=Standalone2026.03 | 73 | 79.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | 79.8 | 78.4 | — | — | — | — | — | |
| PropRAGAvg. Rank=3.82026.05 | 73 | 79.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GFM-RAGEvaluation Protocol=+ IRCoT2026.03 | 72.5 | 80.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | 78 | 77.1 | — | — | — | — | — | |
| GFM-RAG + IRCoTAvg. Rank=3.72026.05 | 72.5 | 80.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SAGEvariant=+ IRCoT, Avg. Rank=3.32026.05 | 72.2 | 82.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Search-R1Model Backbone=Qwen3-8B2026.03 | 72.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CoopRAGBackbone=Gemma2-27B2025.12 | 71.7 | 79 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CoopRAGBackbone=GPT-4o-mini2025.12 | 71.7 | 79.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SAGEvariant=+1 round, Avg. Rank=4.22026.05 | 71.7 | 79.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CoopRAGBackbone=Llama3.3-70B2025.12 | 71.2 | 78.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SubgraphRAGEvaluation Protocol=Standalone2026.03 | 71 | 76.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | 78.3 | 76.7 | — | — | — | — | — | |
| SubgraphRAGAvg. Rank=7.52026.05 | 71 | 76.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Search-R1Model Backbone=Qwen3-4B2026.03 | 70.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CoopRAGBackbone=Gemma2-9B2025.12 | 70 | 78.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GFM-RAGEvaluation Protocol=Standalone2026.03 | 69.8 | 77.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | 79.8 | 78.2 | — | — | — | — | — | |
| GFM-RAGAvg. Rank=7.82026.05 | 69.8 | 77.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BELLEReasoning Strategy=Agent-based Reasoning2025.05 | 69.7 | 75.7 | — | — | 72.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| STRIDEBackbone=Qwen3-8B, Fine-tuning=false2026.04 | 69.2 | 77.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | 81.5 | 75.7 | — | — | — | — | — | |
| GFM-RAGLLM=GPT-4o-mini2026.03 | 67.8 | 76.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| INSES + RouterCategory=Ours2026.03 | 67 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 71 | — | — | — | — | |
| KCSLLM=LLAMA3.1, Diversifying Method=+KCS2025.08 | 66.5 | 73.91 | — | — | — | — | — | — | — | — | — | 69.53 | — | — | 78.1 | 75.2 | 74.25 | — | — | — | — | — | |
| BeamAggRReasoning Strategy=Retrieval-augmented Reasoning2025.05 | 66.1 | 71.6 | — | — | 69.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| STRIDEBackbone=GPT-4o-mini, Fine-tuning=false2026.04 | 66.1 | 77.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | 83.7 | 74.4 | — | — | — | — | — | |
| DualRAGBackbone=GPT-4o-mini, Fine-tuning=false2026.04 | 66 | 72 | — | — | — | — | — | — | — | — | — | — | — | — | — | 75.2 | 71.1 | — | — | — | — | — | |
| C2RAGLLM=GPT-4o-mini2026.03 | 65.9 | 73.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| HippoRAG2Backbone=Llama3.3-70B2025.12 | 65 | 71 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SAGEAvg. Rank=7.52026.05 | 64.5 | 75.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ProbTreeReasoning Strategy=Retrieval-augmented Reasoning2025.05 | 64.3 | 67.9 | — | — | 65.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| NeuroPathLLM=GPT-4o-mini2026.03 | 63.4 | 73.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DeepSieve (Naive RAG)Backbone LLM=DeepSeek-V3, Retriever=DeepSieve (Naive RAG)2025.07 | 62.8 | 68.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SubgraphRAGLLM=GPT-4o-mini2026.03 | 62.7 | 69 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Rejection SamplingModel Backbone=Qwen3-8B2026.03 | 62.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DualRAGBackbone=Qwen3-8B, Fine-tuning=false2026.04 | 62.2 | 69.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | 72.4 | 68.2 | — | — | — | — | — | |
| GenGroundBackbone=Qwen3-8B, Fine-tuning=false2026.04 | 62.1 | 71.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | 76 | 70.2 | — | — | — | — | — | |
| HopRAGBackbone=GPT-4o-mini2025.12 | 61.1 | 68.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| HippoRAG2Backbone=GPT-4o-mini2025.12 | 60.5 | 69.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Composition SamplingLLM=LLAMA3.1, Diversifying Method=+Composition2025.08 | 60.5 | 66.92 | — | — | — | — | — | — | — | — | — | 63.91 | — | — | 72.45 | 67.75 | 67.45 | — | — | — | — | — | |
| LONGAGENTReasoning Strategy=Agent-based Reasoning2025.05 | 60.1 | 65.6 | — | — | 62.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SiReRAGBackbone=GPT-4o-mini2025.12 | 59.6 | 67.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Non-compressionCategory=Non-compression, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 59.5 | 66.85 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 1 | |
| BRIEF-PRO-AUTOCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 59.5 | 66.94 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 23 | |
| BRIEF-PRO-LOWCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 59.5 | 68.31 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 15 | |
| HippoRAGLLM=GPT-4o-mini2026.03 | 59.1 | 69.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RopMuraReasoning Strategy=Agent-based Reasoning2025.05 | 58.8 | 63.2 | — | — | 64 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Rejection SamplingModel Backbone=Qwen3-4B2026.03 | 58.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LiR3AGBackbone=8B, Reasoning=no-think2025.12 | 58.6 | 65.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BRIEF-PRO-MEDIUMCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 58.5 | 67.38 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 26 | |
| RAgent-QwQ-32BReasoning Protocol=Retrieval-augmented Reasoning, Parameter Count=32B2025.01 | 58.4 | 71.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FLAREReasoning Strategy=Retrieval-augmented Reasoning2025.05 | 58.2 | 60.1 | — | — | 63.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Search-o1Reasoning Protocol=Retrieval-augmented Reasoning with Reason-in-Documents, Parameter Count=32B2025.01 | 58 | 71.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LightRAGCategory=Graph-based2026.03 | 58 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 58 | — | — | — | — | |
| Vanilla RAGBackbone=32B, Reasoning=think2025.12 | 57.9 | 63.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Rerank Top-5Category=Extractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 57.5 | 65.78 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 1.8 | |
| ARISEBackbone=Qwen2.5-14B-Instruct, Method Strategy=Search-based2025.04 | 56.5 | 62.61 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Vanilla RAGBackbone=8B, Reasoning=think2025.12 | 56.2 | 60.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Vanilla RAGBackbone=14B, Reasoning=think2025.12 | 56.2 | 60.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LongLLMLinguaCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 55.5 | 64.96 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 5 | |
| RAgent-Qwen2.5-32BReasoning Protocol=Retrieval-augmented Reasoning, Parameter Count=32B2025.01 | 55.4 | 69.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RASTLLM=LLAMA3.1, Diversifying Method=+RAST2025.08 | 55 | 62.88 | — | — | — | — | — | — | — | — | — | 57.83 | — | — | 67.65 | 63.75 | 63.5 | — | — | — | — | — | |
| GPT4LLM=LLAMA3.1, Diversifying Method=+GPT42025.08 | 55 | 62.51 | — | — | — | — | — | — | — | — | — | 58.28 | — | — | 67.6 | 63.95 | 62.62 | — | — | — | — | — | |
| BRIEF-PRO-AUTO_L7CCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 54.5 | 62.68 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 14 | |
| BRIEF-PRO-HIGHCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 54 | 60.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 45 | |
| BRIEF-PRO-MEDIUMCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 53.5 | 62.08 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 26 | |
| IRCoT + IndexRAGCall type=Multi-call, kb=32026.03 | 53.4 | 61.2 | — | — | 61.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Auto-RAGBackbone=Qwen2.5-14B-Instruct, Method Strategy=Prompt-based2025.04 | 53 | 55.13 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RAPTORCategory=Graph-based2026.03 | 52 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 47 | — | — | — | — | |
| EXITCategory=Extractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 52 | 58.93 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 16 | |
| BRIEF-PRO-LOWCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 52 | 60.22 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 15 | |
| Search-o1Model Backbone=Qwen3-8B2026.03 | 51.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| HippoRAG2LLM=GPT-4o-mini2026.03 | 51.5 | 60.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| KCSLLM=QWEN2.5, Diversifying Method=+KCS2025.08 | 51.5 | 56.35 | — | — | — | — | — | — | — | — | — | 51.44 | — | — | 64 | 57.16 | 56.19 | — | — | — | — | — | |
| GPT-4.1-nanoCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 51.5 | 63.79 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 108 | |
| MemSearcherParameter Scale=14B2025.11 | 51.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| R1-SearcherParameter Scale=7B, Search Environment=Google Web Search2025.11 | 51.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Iter-RetGenLLM=GPT-4o-mini2026.03 | 51 | 63.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GPT-4.1-nanoCategory=Abstractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 51 | 64.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 108 | |
| BRIEF-PRO-AUTOCategory=Abstractive, Reader Model (M)=GPT-4.1-nano2025.10 | 51 | 58.68 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 23 | |
| Iter-RetGenToken / Sample=4.7K2026.02 | 50.8 | 59.73 | — | — | 61.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| IRCOTReasoning Strategy=Retrieval-augmented Reasoning2025.05 | 50.7 | 56.8 | — | — | 52.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Search-o1Model Backbone=Qwen3-4B2026.03 | 50.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Vanilla RAGBackbone=14B, Reasoning=no-think2025.12 | 50.2 | 53.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| HippoRAGCall type=Multi-call2026.03 | 50.2 | 57.2 | — | — | 55.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| IQATR (Ours)Backbone=Llama-8B, Retriever=Contriever, Scoring=ReSCORE2025.05 | 50 | 59.7 | 57 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GraphRAGCategory=Graph-based, Top-k=102026.03 | 50 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 43 | — | — | — | — | |
| FastGraphRAGCall type=Single-call2026.03 | 49.5 | 57.4 | — | — | 54.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Rerank Top-3Category=Extractive, Reader Model (M)=Llama-3.1-70B-Instruct2025.10 | 49.5 | 58.75 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 3 | |
| DeepSieve (GraphRAG)Backbone LLM=DeepSeek-V3, Retriever=DeepSieve (GraphRAG)2025.07 | 49.2 | 53.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Agentic-RSearch Agent=R1-Searcher, Search Agent Domain=out-of-domain2026.01 | 49.07 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OriginalLLM=LLAMA3.1, Diversifying Method=ORI2025.08 | 49 | 57.64 | — | — | — | — | — | — | — | — | — | 51.36 | — | — | 62.65 | 58.54 | 58.32 | — | — | — | — | — | |
| BRIEF-PRO-LOWCategory=Abstractive, Reader Model (M)=Llama-3.1-8B-Instruct2025.10 | 49 | 58.39 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 15 |