Multi-hop Question Answering on HotpotQA (EM, F1)
74.9F1CIRAG
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CIRAGBase LLM=Qwen2.5-max2026.01 | 74.9 | 60.9 | |
| MetaRAGBase LLM=Qwen2.5-max2026.01 | 74.6 | 60.8 | |
| DualRAGBase LLM=Qwen2.5-max2026.01 | 73.3 | 57.8 | |
| KiRAGBase LLM=Qwen2.5-max2026.01 | 73.2 | 59 | |
| IRCOTBase LLM=Qwen2.5-max2026.01 | 72.8 | 58.1 | |
| GraphAnchorBackbone=Qwen2.5-14B-Instruct2026.01 | 69.64 | 57.44 | |
| GraphAnchorBackbone=Qwen3-32B2026.01 | 69.34 | 55.85 | |
| CIRAGBase LLM=Qwen2.5-7B2026.01 | 67.1 | 52.5 | |
| NativeRAGBase LLM=Qwen2.5-max2026.01 | 66.9 | 52.4 | |
| GraphAnchorBackbone=Qwen2.5-7B-Instruct2026.01 | 66.03 | 53.8 | |
| GraphAnchorBackbone=Llama3.1-8B-Instruct2026.01 | 64.37 | 50.1 | |
| MetaRAGBase LLM=Qwen2.5-7B2026.01 | 63.3 | 49.6 | |
| DualRAG-FTBase LLM=Qwen2.5-7B2026.01 | 62.6 | 47.1 | |
| KiRAGBase LLM=Qwen2.5-7B2026.01 | 62.1 | 49 | |
| DeepNoteBackbone=Qwen2.5-7B-Instruct2026.01 | 59.97 | 48.6 | |
| DeepNoteBackbone=Llama3.1-8B-Instruct2026.01 | 59.51 | 47.8 | |
| Iter-RetGenBackbone=Qwen2.5-7B-Instruct2026.01 | 58.81 | 47.4 | |
| DualRAGBase LLM=Qwen2.5-7B2026.01 | 58.7 | 44.8 | |
| IRCOTBase LLM=Qwen2.5-7B2026.01 | 56.8 | 42.4 | |
| FLAREBase LLM=Qwen2.5-7B2026.01 | 56.4 | 41.9 | |
| Iter-RetGenBackbone=Llama3.1-8B-Instruct2026.01 | 55.97 | 43.8 | |
| HippoRAG2026.02 | 55.87 | 49.5 | |
| Search-R1Backbone=Qwen2.5-7B-Instruct2026.01 | 54.78 | 43.4 | |
| Vanilla RAGBackbone=Qwen2.5-14B-Instruct2026.01 | 54.73 | 43.8 | |
| Vanilla RAGBackbone=Qwen3-32B2026.01 | 54.32 | 42 | |
| QuCo-RAGBackbone=OLMo-2-32B2025.12 | 54.2 | 41.6 | |
| Vanilla RAGBackbone=Llama3.1-8B-Instruct2026.01 | 52.94 | 42.2 | |
| LiR3AGBackbone=8B, Reasoning=no-think2025.12 | 52.1 | 40.2 | |
| Vanilla RAGBackbone=Qwen2.5-7B-Instruct2026.01 | 50.93 | 40.4 | |
| IRCoTBackbone=Llama3.1-8B-Instruct2026.01 | 50.41 | 39.6 | |
| NativeRAGBase LLM=Qwen2.5-7B2026.01 | 50.2 | 34.5 | |
| Vanilla RAGBackbone=32B, Reasoning=think2025.12 | 50.1 | 38 | |
| IRCoTBackbone=Qwen2.5-7B-Instruct2026.01 | 49.55 | 36.6 | |
| QuCo-RAGBackbone=OLMo-2-13B2025.12 | 46.8 | 35 | |
| QuCo-RAGBackbone=OLMo-2-7B2025.12 | 46.1 | 35.3 | |
| Vanilla RAGBackbone=14B, Reasoning=think2025.12 | 45.7 | 35.6 | |
| Vanilla RAGBackbone=32B, Reasoning=no-think2025.12 | 45.2 | 34.6 | |
| Vanilla RAGBackbone=8B, Reasoning=think2025.12 | 44.5 | 32.8 | |
| Vanilla RAGBackbone=14B, Reasoning=no-think2025.12 | 44.1 | 34.2 | |
| HyperRetriever2026.02 | 43.65 | 42.5 | |
| REPAIR-7BAnswer generator=Llama-3.1-8B-Instruct, Context size=top-10, Retrieval protocol=rerank top-100 Contriever2026.01 | 43 | 32.2 | |
| HyperGraphRAG2026.02 | 42.69 | 51 | |
| ETCBackbone=OLMo-2-32B2025.12 | 42.2 | 30.8 | |
| REARANK-7BAnswer generator=Llama-3.1-8B-Instruct, Context size=top-10, Retrieval protocol=rerank top-100 Contriever2026.01 | 42 | 30.2 | |
| RAPTOR2026.02 | 41.56 | 35.5 | |
| HyperMemorybeam width=3, depth=32026.02 | 41.51 | 35.5 | |
| SR-RAGBackbone=OLMo-2-7B2025.12 | 40.7 | 29.7 | |
| SR-RAGBackbone=OLMo-2-32B2025.12 | 40.4 | 29.5 | |
| SeaKRBackbone=OLMo-2-32B2025.12 | 40.4 | 28.7 | |
| Vanilla RAGBackbone=8B, Reasoning=no-think2025.12 | 40.3 | 31 | |
| FLAREBackbone=OLMo-2-32B2025.12 | 39.8 | 28.3 | |
| SR-RAGBackbone=OLMo-2-13B2025.12 | 39.5 | 29.7 | |
| ETCBackbone=OLMo-2-13B2025.12 | 39.5 | 29.3 | |
| SUNAR_RAnswer generator=Llama-3.1-8B-Instruct, Context size=top-10, Retrieval protocol=rerank top-100 Contriever2026.01 | 39.4 | 28.2 | |
| RankZephyr-7BAnswer generator=Llama-3.1-8B-Instruct, Context size=top-10, Retrieval protocol=rerank top-100 Contriever2026.01 | 38.8 | 28.2 | |
| QwenRerank-7BAnswer generator=Llama-3.1-8B-Instruct, Context size=top-10, Retrieval protocol=rerank top-100 Contriever2026.01 | 37.9 | 27.8 | |
| ContrieverAnswer generator=Llama-3.1-8B-Instruct, Context size=top-10, Retrieval protocol=initial retrieval2026.01 | 37.4 | 26.4 | |
| SeaKRBackbone=OLMo-2-13B2025.12 | 37.3 | 26.2 | |
| Rank-R1-7BAnswer generator=Llama-3.1-8B-Instruct, Context size=top-10, Retrieval protocol=rerank top-100 Contriever2026.01 | 37.1 | 27 | |
| Direct OutputBackbone=32B, Reasoning=think2025.12 | 35.8 | 25.4 | |
| SeaKRBackbone=OLMo-2-7B2025.12 | 35 | 24.8 | |
| ETCBackbone=OLMo-2-7B2025.12 | 34.7 | 25.1 | |
| Wo-RAGBackbone=OLMo-2-13B2025.12 | 33.6 | 24.4 | |
| Wo-RAGBackbone=OLMo-2-7B2025.12 | 31.6 | 22.6 | |
| Direct OutputBackbone=14B, Reasoning=think2025.12 | 31.6 | 22.8 | |
| Wo-RAGBackbone=OLMo-2-32B2025.12 | 31.3 | 22 | |
| Direct OutputBackbone=8B, Reasoning=think2025.12 | 28.5 | 18.5 | |
| FLAREBackbone=OLMo-2-7B2025.12 | 28.4 | 20.3 | |
| DRAGINBackbone=OLMo-2-13B2025.12 | 27.6 | 19.5 | |
| Direct OutputBackbone=8B, Reasoning=no-think2025.12 | 25.7 | 17.2 | |
| DRAGINBackbone=OLMo-2-7B2025.12 | 24.7 | 17.5 | |
| Direct OutputBackbone=32B, Reasoning=no-think2025.12 | 24.6 | 17.6 | |
| DRAGINBackbone=OLMo-2-32B2025.12 | 24.3 | 17.7 | |
| Direct OutputBackbone=14B, Reasoning=no-think2025.12 | 23.6 | 15 | |
| FS-RAGBackbone=OLMo-2-13B2025.12 | 21.9 | 14.6 | |
| FLAREBackbone=OLMo-2-13B2025.12 | 21.9 | 15.3 | |
| FS-RAGBackbone=OLMo-2-7B2025.12 | 20.7 | 14.5 | |
| FS-RAGBackbone=OLMo-2-32B2025.12 | 19.5 | 13.9 | |
| ToG2026.02 | 11 | 10.08 |