Multi-hop Question Answering on 2Wiki (test)
69.7F1 ScoreCatRAG
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| CatRAGQA reader=Llama-3.3-70B-Instruct2026.02 | 69.7 | — | — | — | — | — | |
| HippoRAG 2QA reader=Llama-3.3-70B-Instruct, Retrieval=Structure-Aware RAG2026.02 | 68.1 | — | — | — | — | — | |
| text-embedding-3-smallQA reader=Llama-3.3-70B-Instruct, Retrieval=Standard Retrieval2026.02 | 56.9 | — | — | — | — | — | |
| RAPTORQA reader=Llama-3.3-70B-Instruct, Retrieval=Structure-Aware RAG2026.02 | 56.7 | — | — | — | — | — | |
| GTR (T5-base)QA reader=Llama-3.3-70B-Instruct, Retrieval=Standard Retrieval2026.02 | 52.8 | — | — | — | — | — | |
| LightRAGQA reader=Llama-3.3-70B-Instruct, Retrieval=Structure-Aware RAG2026.02 | 49.7 | — | — | — | — | — | |
| OThink-SRR1Base model=Qwen2.5-7B-Instruct2026.03 | 49.33 | — | — | 41.52 | — | — | |
| ReSearchBase model=Qwen2.5-7B-Instruct2026.03 | 48.66 | — | — | 40.6 | — | — | |
| OThink-SRR1Base model=Qwen2.5-3B-Instruct2026.03 | 43.54 | — | — | 35.27 | — | — | |
| NoneQA reader=Llama-3.3-70B-Instruct, Retrieval=None2026.02 | 42.8 | — | — | — | — | — | |
| Search-R1 + EKABackbone=Qwen2.5-7B-Instruct2025.12 | 42.7 | — | — | — | — | — | |
| ContrieverQA reader=Llama-3.3-70B-Instruct, Retrieval=Standard Retrieval2026.02 | 41.9 | — | — | — | — | — | |
| Search-R1Backbone=Qwen2.5-7B-Instruct2025.12 | 41.4 | — | — | — | — | — | |
| BM25QA reader=Llama-3.3-70B-Instruct, Retrieval=Standard Retrieval2026.02 | 39.9 | — | — | — | — | — | |
| Search-R1Base model=Qwen2.5-7B-Instruct2026.03 | 39.03 | — | — | 33.08 | — | — | |
| Search-R1Base model=Qwen2.5-3B-Instruct2026.03 | 37.87 | — | — | 32.11 | — | — | |
| Iter-RetGenBase model=Qwen2.5-7B-Instruct2026.03 | 34.17 | — | — | 27.85 | — | — | |
| ReSearchBase model=Qwen2.5-3B-Instruct2026.03 | 32.22 | — | — | 26.82 | — | — | |
| Basic RAGBase model=Qwen2.5-7B-Instruct2026.03 | 32.01 | — | — | 25.59 | — | — | |
| Iter-RetGenBase model=Qwen2.5-3B-Instruct2026.03 | 31.47 | — | — | 25.97 | — | — | |
| No RAGBase model=Qwen2.5-7B-Instruct2026.03 | 30.32 | — | — | 25.52 | — | — | |
| Basic RAGBase model=Qwen2.5-3B-Instruct2026.03 | 30.18 | — | — | 24.54 | — | — | |
| IRCOTBase model=Qwen2.5-3B-Instruct2026.03 | 30 | — | — | 21.45 | — | — | |
| Rejection SamplingBackbone=Qwen2.5-7B-Instruct2025.12 | 29.6 | — | — | — | — | — | |
| IRCOTBase model=Qwen2.5-7B-Instruct2026.03 | 29.28 | — | — | 21.49 | — | — | |
| R1-instructBackbone=Qwen2.5-7B-Instruct2025.12 | 29.2 | — | — | — | — | — | |
| No RAGBase model=Qwen2.5-3B-Instruct2026.03 | 28.73 | — | — | 24.79 | — | — | |
| R1-baseBackbone=Qwen2.5-7B-Instruct2025.12 | 27.3 | — | — | — | — | — | |
| SFTBackbone=Qwen2.5-7B-Instruct2025.12 | 25.9 | — | — | — | — | — | |
| Direct InferenceBackbone=Qwen2.5-7B-Instruct2025.12 | 25 | — | — | — | — | — | |
| Standard RAGBackbone=Qwen2.5-7B-Instruct2025.12 | 23.5 | — | — | — | — | — | |
| Search-o1Backbone=Qwen2.5-7B-Instruct2025.12 | 17.6 | — | — | — | — | — | |
| IRCoTBackbone=Qwen2.5-7B-Instruct2025.12 | 14.9 | — | — | — | — | — | |
| CoTBackbone=Qwen2.5-7B-Instruct2025.12 | 11.1 | — | — | — | — | — | |
| ChainRAG (AnsInt)Backbone=Qwen2.5, Model Size=14B2025.02 | — | 0.5971 | — | — | — | — | |
| ChainRAG (AnsInt)Backbone=Qwen2.5, Model Size=7B2025.02 | — | 0.4825 | — | — | — | — | |
| ChainRAG (CxtInt)Backbone=Qwen2.5, Model Size=14B2025.02 | — | 0.6314 | — | — | — | — | |
| ChainRAG (CxtInt)Backbone=Qwen2.5, Model Size=7B2025.02 | — | 0.5218 | — | — | — | — | |
| HippoRAG w/ IRCoTBackbone=Qwen2.5, Model Size=14B2025.02 | — | 0.585 | — | — | — | — | |
| HippoRAG w/ IRCoTBackbone=Qwen2.5, Model Size=7B2025.02 | — | 0.5117 | — | — | — | — | |
| LongRAGBackbone=Qwen2.5, Model Size=14B2025.02 | — | 0.5495 | — | — | — | — | |
| LongRAGBackbone=Qwen2.5, Model Size=7B2025.02 | — | 0.4902 | — | — | — | — | |
| Telegraph English (TE)Comparison Baseline=Full-NL2026.06 | — | — | — | — | 1.53 | 0.17 | |
| Telegraph English (TE)Comparison Baseline=LLMLingua-2, Budget Regime=rate-502026.06 | — | — | — | — | 1.6 | 0.13 |