Multi-hop Question Answering on HotpotQA
77.4F1 ScoreSAFE
Evaluation results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SAFEBackbone=Qwen 3 8B2026.04 | 77.4 | 89.4 | 61.6 | — | — | — | — | — | — | — | — | — | — | — | |
| SAFEBackbone=Qwen 2.5 14B2026.04 | 76.8 | 91.2 | 60.8 | — | — | — | — | — | — | — | — | — | — | — | |
| SAFEBackbone=Gemma 3 12B2026.04 | 75.8 | 91.2 | 58.9 | — | — | — | — | — | — | — | — | — | — | — | |
| SAFEBackbone=Qwen 3 4B2026.04 | 75.7 | 89 | 60.5 | — | — | — | — | — | — | — | — | — | — | — | |
| ARISEBackbone=Qwen2.5-14B-Instruct, Method Strategy=Search-based2025.04 | 75.39 | — | 73.5 | — | — | — | — | — | — | — | — | — | — | — | |
| ARISEBackbone=Llama3.1-8B-Instruct, Method Strategy=Search-based2025.04 | 74.78 | — | 63 | — | — | — | — | — | — | — | — | — | — | — | |
| BubbleRAGModel Scale=30B2026.03 | 74.35 | — | — | — | — | — | — | — | — | — | — | — | — | 82.6 | |
| RATTBackbone=Qwen2.5-14B-Instruct, Method Strategy=Search-based2025.04 | 73.91 | — | 64.5 | — | — | — | — | — | — | — | — | — | — | — | |
| ARISEBackbone=Qwen2.5-7B-Instruct, Method Strategy=Search-based2025.04 | 73.87 | — | 66.5 | — | — | — | — | — | — | — | — | — | — | — | |
| HippoRAG2Model Scale=8B2026.03 | 73.33 | — | — | — | — | — | — | — | — | — | — | — | — | 79 | |
| Naive RAGModel Scale=30B2026.03 | 73.07 | — | — | — | — | — | — | — | — | — | — | — | — | 80.5 | |
| C2RAGLLM=GPT-4o-mini2026.03 | 72.8 | — | 55.3 | — | — | — | — | — | — | — | — | — | — | — | |
| No FeedbackBackbone=Qwen 3 4B2026.04 | 72.4 | 86.5 | 56.2 | — | — | — | — | — | — | — | — | — | — | — | |
| SAFEBackbone=Llama 3.1 8B2026.04 | 72.2 | 90.1 | 57.1 | — | — | — | — | — | — | — | — | — | — | — | |
| BubbleRAGModel Scale=8B2026.03 | 71.82 | — | — | — | — | — | — | — | — | — | — | — | — | 79.1 | |
| HippoRAG2Model Scale=30B2026.03 | 71.72 | — | — | — | — | — | — | — | — | — | — | — | — | 80.2 | |
| CoT-SCBackbone=Llama3.1-8B-Instruct, Method Strategy=Search-based2025.04 | 71.4 | — | 64.5 | — | — | — | — | — | — | — | — | — | — | — | |
| CoT-SCBackbone=Qwen2.5-14B-Instruct, Method Strategy=Search-based2025.04 | 71.39 | — | 63 | — | — | — | — | — | — | — | — | — | — | — | |
| Self-FeedbackBackbone=Qwen 3 8B2026.04 | 71.2 | 84.4 | 55 | — | — | — | — | — | — | — | — | — | — | — | |
| RATTBackbone=Llama3.1-8B-Instruct, Method Strategy=Search-based2025.04 | 71.18 | — | 58.5 | — | — | — | — | — | — | — | — | — | — | — | |
| Self-FeedbackBackbone=Llama 3.1 8B2026.04 | 70.8 | 83.8 | 54.9 | — | — | — | — | — | — | — | — | — | — | — | |
| CoT-SCBackbone=Qwen2.5-7B-Instruct, Method Strategy=Search-based2025.04 | 70.66 | — | 60.5 | — | — | — | — | — | — | — | — | — | — | — | |
| Self-FeedbackBackbone=Qwen 3 4B2026.04 | 70.6 | 82.6 | 56.5 | — | — | — | — | — | — | — | — | — | — | — | |
| HippoRAGCall type=Multi-call2026.03 | 70.5 | 60.7 | 56.5 | — | — | — | — | — | — | — | — | — | — | — | |
| Self-FeedbackBackbone=Gemma 3 12B2026.04 | 70.5 | 83.4 | 54.6 | — | — | — | — | — | — | — | — | — | — | — | |
| SARGBackbone=GPT-4o-mini2025.06 | 70.19 | — | 57.9 | — | — | — | — | — | — | — | — | — | — | — | |
| Verify-and-EditBackbone=Qwen2.5-14B-Instruct, Method Strategy=Prompt-based2025.04 | 70.16 | — | 62.5 | — | — | — | — | — | — | — | — | — | — | — | |
| TRACEBackbone=Llama 3 8B2026.04 | 70 | — | 55.1 | — | — | — | — | — | — | — | — | — | — | — | |
| Verify-and-EditBackbone=Qwen2.5-7B-Instruct, Method Strategy=Prompt-based2025.04 | 69.87 | — | 67 | — | — | — | — | — | — | — | — | — | — | — | |
| Self-FeedbackBackbone=Qwen 2.5 14B2026.04 | 69.7 | 84.3 | 53.9 | — | — | — | — | — | — | — | — | — | — | — | |
| Self-RAG 7BRetrieval-Augmented Generation=Enabled2024.07 | 69.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| IndexRAGCall type=Single-call, kb=32026.03 | 68.9 | 59.3 | 54.1 | — | — | — | — | — | — | — | — | — | — | — | |
| RATTBackbone=Qwen2.5-7B-Instruct, Method Strategy=Search-based2025.04 | 68.88 | — | 58.5 | — | — | — | — | — | — | — | — | — | — | — | |
| IN-CONTEXT2026.02 | 68.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| IRCoT + IndexRAGCall type=Multi-call, kb=32026.03 | 68.7 | 59.7 | 54.5 | — | — | — | — | — | — | — | — | — | — | — | |
| No FeedbackBackbone=Qwen 2.5 14B2026.04 | 68.4 | 87.9 | 49.5 | — | — | — | — | — | — | — | — | — | — | — | |
| No FeedbackBackbone=Gemma 3 12B2026.04 | 68.1 | 86.6 | 51.1 | — | — | — | — | — | — | — | — | — | — | — | |
| GFM-RAGLLM=GPT-4o-mini2026.03 | 68 | — | 51.2 | — | — | — | — | — | — | — | — | — | — | — | |
| Query2DocBackbone=Qwen2.5-14B-Instruct, Method Strategy=Prompt-based2025.04 | 67.65 | — | 61 | — | — | — | — | — | — | — | — | — | — | — | |
| Naive RAGModel Scale=8B2026.03 | 67.41 | — | — | — | — | — | — | — | — | — | — | — | — | 75 | |
| Query2DocBackbone=Qwen2.5-7B-Instruct, Method Strategy=Prompt-based2025.04 | 67.31 | — | 60 | — | — | — | — | — | — | — | — | — | — | — | |
| RAPTORModel Scale=30B2026.03 | 66.75 | — | — | — | — | — | — | — | — | — | — | — | — | 68 | |
| No FeedbackBackbone=Llama 3.1 8B2026.04 | 66.7 | 78.9 | 51.1 | — | — | — | — | — | — | — | — | — | — | — | |
| Auto-RAGBackbone=Qwen2.5-14B-Instruct, Method Strategy=Prompt-based2025.04 | 66.64 | — | 68 | — | — | — | — | — | — | — | — | — | — | — | |
| BELLEReasoning Strategy=Agent-based Reasoning2025.05 | 66.5 | 63.7 | 59.2 | — | — | — | — | — | — | — | — | — | — | — | |
| Auto-RAGBackbone=Qwen2.5-7B-Instruct, Method Strategy=Prompt-based2025.04 | 66.33 | — | 66.5 | — | — | — | — | — | — | — | — | — | — | — | |
| No FeedbackBackbone=Qwen 3 8B2026.04 | 65.8 | 87.4 | 47.5 | — | — | — | — | — | — | — | — | — | — | — | |
| IRCoTBackbone=Llama 3 8B2026.04 | 65.7 | — | 50.8 | — | — | — | — | — | — | — | — | — | — | — | |
| Verify-and-EditBackbone=Llama3.1-8B-Instruct, Method Strategy=Prompt-based2025.04 | 65.07 | — | 50.5 | — | — | — | — | — | — | — | — | — | — | — | |
| LogicGazeBackbone=LLaMA3-8B, Retrieval Setting=Yes, Evaluation Protocol=RAG2026.01 | 64.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| NeuroPathLLM=GPT-4o-mini2026.03 | 64.9 | — | 51.5 | — | — | — | — | — | — | — | — | — | — | — | |
| Self-AskBackbone=Qwen2.5-14B-Instruct, Method Strategy=Prompt-based2025.04 | 64.74 | — | 58.5 | — | — | — | — | — | — | — | — | — | — | — | |
| LongRAGLLM=GPT4o-mini2025.02 | 64.74 | — | 51 | — | — | — | — | — | — | — | — | — | — | — | |
| HippoRAG2LLM=GPT-4o-mini2026.03 | 64.7 | — | 50.7 | — | — | — | — | — | — | — | — | — | — | — | |
| ChainRAG (CxtInt)LLM=GPT4o-mini, Answering Strategy=CxtInt2025.02 | 64.59 | — | 50 | — | — | — | — | — | — | — | — | — | — | — | |
| ChainRAG (CxtInt)LLM=Qwen2.5-72B, Answering Strategy=CxtInt2025.02 | 64.54 | — | 52 | — | — | — | — | — | — | — | — | — | — | — | |
| RAGBackbone=LLaMA-3.1-8B-Instruct2025.06 | 64.5 | — | 44.9 | — | — | — | — | — | — | — | — | — | — | — | |
| IterDRAGBackbone=LLaMA3-8B, Retrieval Setting=Yes, Evaluation Protocol=RAG2026.01 | 64.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ChainRAG (CxtInt)LLM=GLM-4-Plus, Answering Strategy=CxtInt2025.02 | 64.22 | — | 50 | — | — | — | — | — | — | — | — | — | — | — | |
| SARGBackbone=Qwen2.5-7B-Instruct2025.06 | 64.13 | — | 51.24 | — | — | — | — | — | — | — | — | — | — | — | |
| HippoRAG w/ IRCoTLLM=GLM-4-Plus2025.02 | 63.91 | — | 48 | — | — | — | — | — | — | — | — | — | — | — | |
| VanillaBackbone=Qwen2.5-14B-Instruct, Method Strategy=Vanilla2025.04 | 63.63 | — | 59.5 | — | — | — | — | — | — | — | — | — | — | — | |
| VanillaBackbone=Qwen2.5-7B-Instruct, Method Strategy=Vanilla2025.04 | 63.63 | — | 54.5 | — | — | — | — | — | — | — | — | — | — | — | |
| VanillaBackbone=Llama3.1-8B-Instruct, Method Strategy=Vanilla2025.04 | 63.63 | — | 55.5 | — | — | — | — | — | — | — | — | — | — | — | |
| RankRAGBackbone=LLaMA3-8B, Retrieval Setting=Yes, Evaluation Protocol=RAG2026.01 | 63.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Naive RAGCall type=Single-call2026.03 | 63.6 | 54 | 50.2 | — | — | — | — | — | — | — | — | — | — | — | |
| RAPTORCall type=Single-call2026.03 | 63.6 | 54.1 | 50.2 | — | — | — | — | — | — | — | — | — | — | — | |
| Query2DocBackbone=Llama3.1-8B-Instruct, Method Strategy=Prompt-based2025.04 | 63.52 | — | 57.5 | — | — | — | — | — | — | — | — | — | — | — | |
| FastGraphRAGCall type=Single-call2026.03 | 63.5 | 53.8 | 50 | — | — | — | — | — | — | — | — | — | — | — | |
| ShardMemoModel=GPT-OSS-120B, Context Budget=56K2026.01 | 63.41 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SentGraphRetrieval=BGE, Retrieval Units=Sentence, Avg # Units=2.72026.01 | 62.92 | — | 48.8 | — | — | — | — | — | — | — | — | — | — | — | |
| BeamAggRReasoning Strategy=Retrieval-augmented Reasoning2025.05 | 62.9 | 59.2 | 55.6 | — | — | — | — | — | — | — | — | — | — | — | |
| Clue-RAGModel Scale=30B2026.03 | 62.87 | — | — | — | — | — | — | — | — | — | — | — | — | 67.3 | |
| AutoRAGBackbone=LLaMA3-8B, Retrieval Setting=Yes, Evaluation Protocol=RAG2026.01 | 62.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Iter-RetGenLLM=GPT-4o-mini2026.03 | 62.7 | — | 49.1 | — | — | — | — | — | — | — | — | — | — | — | |
| RAPTORModel Scale=8B2026.03 | 62.66 | — | — | — | — | — | — | — | — | — | — | — | — | 68.6 | |
| IRCoTCall type=Multi-call2026.03 | 62.5 | 54.9 | 49.8 | — | — | — | — | — | — | — | — | — | — | — | |
| Self-AskBackbone=Llama3.1-8B-Instruct, Method Strategy=Prompt-based2025.04 | 62.1 | — | 57 | — | — | — | — | — | — | — | — | — | — | — | |
| GAMModel=GPT-OSS-120B, Context Budget=56K2026.01 | 62.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SARGBackbone=LLaMA-3.1-8B-Instruct2025.06 | 61.99 | — | 48.66 | — | — | — | — | — | — | — | — | — | — | — | |
| RQ-RAGBackbone=LLaMA3-8B, Retrieval Setting=Yes, Evaluation Protocol=RAG2026.01 | 61.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ShardMemoModel=GPT-OSS-120B, Context Budget=224K2026.01 | 61.88 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LongRAGLLM=GLM-4-Plus2025.02 | 61.6 | — | 48.5 | — | — | — | — | — | — | — | — | — | — | — | |
| Self-AskBackbone=Qwen2.5-7B-Instruct, Method Strategy=Prompt-based2025.04 | 61.43 | — | 44 | — | — | — | — | — | — | — | — | — | — | — | |
| ITER-RETGENLLM=GLM-4-Plus, Iterations=32025.02 | 61.03 | — | 48.5 | — | — | — | — | — | — | — | — | — | — | — | |
| GAMModel=GPT-OSS-120B, Context Budget=224K2026.01 | 60.92 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Q-DREAMBackbone=ChatGPT2025.05 | 60.9 | — | 48.4 | — | — | — | — | — | — | — | — | — | — | — | |
| ChainRAG (AnsInt)LLM=GPT4o-mini, Answering Strategy=AnsInt2025.02 | 60.73 | — | 46 | — | — | — | — | — | — | — | — | — | — | — | |
| ChainRAG (AnsInt)LLM=Qwen2.5-72B, Answering Strategy=AnsInt2025.02 | 60.55 | — | 46 | — | — | — | — | — | — | — | — | — | — | — | |
| ProbTreeReasoning Strategy=Retrieval-augmented Reasoning2025.05 | 60.4 | 60.6 | 56.3 | — | — | — | — | — | — | — | — | — | — | — | |
| BM25Call type=Single-call2026.03 | 60.3 | 51.2 | 47.2 | — | — | — | — | — | — | — | — | — | — | — | |
| LongRAGLLM=Qwen2.5-72B2025.02 | 60.29 | — | 46.5 | — | — | — | — | — | — | — | — | — | — | — | |
| NaiveRAG w/ QDLLM=GPT4o-mini, Question Decomposition=true2025.02 | 60 | — | 43.5 | — | — | — | — | — | — | — | — | — | — | — | |
| IRCoTLLM=GPT-4o-mini2026.03 | 59.8 | — | 44.6 | — | — | — | — | — | — | — | — | — | — | — | |
| HippoRAG2Backbone=GPT-4o-mini2025.06 | 59.43 | — | 40.1 | — | — | — | — | — | — | — | — | — | — | — | |
| IQATR (Ours)Backbone=Llama-8B, Retriever=Contriever, Scoring=ReSCORE2025.05 | 59.3 | — | 47.2 | 59.6 | — | — | — | — | — | — | — | — | — | — | |
| Self-RAGBackbone=LLaMA3-8B, Retrieval Setting=Yes, Evaluation Protocol=RAG2026.01 | 59.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LONGAGENTReasoning Strategy=Agent-based Reasoning2025.05 | 59.2 | 58.1 | 52.4 | — | — | — | — | — | — | — | — | — | — | — | |
| HippoRAGBackbone=GPT 3.5-turbo, Retriever=ColBERTv22025.05 | 59.2 | — | 45.7 | — | — | — | — | — | — | — | — | — | — | — | |
| NaiveRAG w/ QDLLM=GLM-4-Plus, Question Decomposition=true2025.02 | 59.05 | — | 44 | — | — | — | — | — | — | — | — | — | — | — |