Question Answering on 2WikiMQA
74.9F1Oracle
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| OracleBackbone=Llama 3.3 70B, Context Length=1500 words2026.01 | 74.9 | — | — | — | — | — | — | |
| OracleBackbone=Qwen 2.5 72B, Context Length=1500 words2026.01 | 69.9 | — | — | — | — | — | — | |
| DF-RAG + ICBackbone=Qwen 2.5 72B, Context Length=1500 words2026.01 | 69 | — | — | — | — | — | — | |
| OracleBackbone=Llama 3.3 70B, Context Length=1000 words2026.01 | 68.6 | — | — | — | — | — | — | |
| DF-RAG + ICBackbone=Llama 3.3 70B, Context Length=1500 words2026.01 | 68.6 | — | — | — | — | — | — | |
| OracleBackbone=Qwen 2.5 72B, Context Length=1000 words2026.01 | 67.2 | — | — | — | — | — | — | |
| HippoRAG + QREAM-ICLRAG Framework=HippoRAG Integration2026.04 | 66.8 | — | — | — | 39.8 | — | — | |
| HippoRAG + QREAM-FTRAG Framework=HippoRAG Integration2026.04 | 65.6 | — | — | — | 38.5 | — | — | |
| OracleBackbone=Llama 3.3 70B, Context Length=500 words2026.01 | 65.3 | — | — | — | — | — | — | |
| OracleBackbone=Qwen 2.5 72B, Context Length=500 words2026.01 | 64.8 | — | — | — | — | — | — | |
| DF-RAGBackbone=Llama 3.3 70B, Context Length=1500 words2026.01 | 63.9 | — | — | — | — | — | — | |
| HippoRAG 2Setting=Local2026.05 | 63.5 | — | — | — | 57.9 | 3.9 | 1 | |
| RAG with gMMRBackbone=Llama 3.3 70B, Context Length=1500 words2026.01 | 63.4 | — | — | — | — | — | — | |
| DF-RAG + ICBackbone=Llama 3.3 70B, Context Length=1000 words2026.01 | 63.3 | — | — | — | — | — | — | |
| FD-RAG-LocalSetting=Local2026.05 | 63.1 | — | — | — | 59.2 | 0.62 | 0.45 | |
| FD-RAGSetting=Federated2026.05 | 62.6 | — | — | — | 58.9 | 1.28 | 0.68 | |
| DF-RAGBackbone=Qwen 2.5 72B, Context Length=1500 words2026.01 | 62.1 | — | — | — | — | — | — | |
| HyperGraphRAGSetting=Local2026.05 | 61.8 | — | — | — | 57.1 | 4.9 | 1 | |
| LongRAGBackbone=Llama 3.3 70B, Context Length=independent2026.01 | 61.7 | — | — | — | — | — | — | |
| Vanilla RAGBackbone=Llama 3.3 70B, Context Length=1500 words2026.01 | 61.6 | — | — | — | — | — | — | |
| RAPTORSetting=Local2026.05 | 61.2 | — | — | — | 50.6 | 5.9 | 1 | |
| FD-RAG (w/o fusion)Setting=Federated2026.05 | 60.3 | — | — | — | 56.7 | 1.58 | 0.86 | |
| LongRAGBackbone=Qwen 2.5 72B, Context Length=independent2026.01 | 59.9 | — | — | — | — | — | — | |
| RAG with gMMRBackbone=Qwen 2.5 72B, Context Length=1500 words2026.01 | 59.8 | — | — | — | — | — | — | |
| DF-RAG + ICBackbone=Qwen 2.5 72B, Context Length=1000 words2026.01 | 59.7 | — | — | — | — | — | — | |
| Vanilla RAGBackbone=Qwen 2.5 72B, Context Length=1500 words2026.01 | 59.5 | — | — | — | — | — | — | |
| DF-RAGBackbone=Llama 3.3 70B, Context Length=1000 words2026.01 | 58.9 | — | — | — | — | — | — | |
| RAPTORBackbone=Llama 3.3 70B, Context Length=independent2026.01 | 58.9 | — | — | — | — | — | — | |
| GraphRAGSetting=Local2026.05 | 58.9 | — | — | — | 52.7 | 7.4 | 1 | |
| RAG with gMMRBackbone=Llama 3.3 70B, Context Length=1000 words2026.01 | 58.7 | — | — | — | — | — | — | |
| RAPTORBackbone=Qwen 2.5 72B, Context Length=independent2026.01 | 58.4 | — | — | — | — | — | — | |
| Vanilla RAGBackbone=Llama 3.3 70B, Context Length=1000 words2026.01 | 58.2 | — | — | — | — | — | — | |
| DF-RAGBackbone=Llama 3.3 70B, Context Length=500 words2026.01 | 58.1 | — | — | — | — | — | — | |
| LongRAGSetting=Local2026.05 | 58.1 | — | — | — | 54 | 12.3 | 7.35 | |
| DF-RAGBackbone=Qwen 2.5 72B, Context Length=1000 words2026.01 | 57.4 | — | — | — | — | — | — | |
| HippoRAGBackbone=Llama 3.3 70B, Context Length=independent2026.01 | 57.1 | — | — | — | — | — | — | |
| DF-RAG + ICBackbone=Qwen 2.5 72B, Context Length=500 words2026.01 | 56.8 | — | — | — | — | — | — | |
| RAGRouteSetting=Federated2026.05 | 55.9 | — | — | — | 52.8 | 3.2 | 1.38 | |
| RAG with gMMRBackbone=Qwen 2.5 72B, Context Length=1000 words2026.01 | 55.5 | — | — | — | — | — | — | |
| DF-RAG + ICBackbone=Llama 3.3 70B, Context Length=500 words2026.01 | 55.2 | — | — | — | — | — | — | |
| Vanilla RAGBackbone=Qwen 2.5 72B, Context Length=1000 words2026.01 | 55 | — | — | — | — | — | — | |
| RAG with gMMRBackbone=Llama 3.3 70B, Context Length=500 words2026.01 | 54.8 | — | — | — | — | — | — | |
| DF-RAGBackbone=Qwen 2.5 72B, Context Length=500 words2026.01 | 53.7 | — | — | — | — | — | — | |
| HippoRAGBackbone=Qwen 2.5 72B, Context Length=independent2026.01 | 53.2 | — | — | — | — | — | — | |
| RAG with gMMRBackbone=Qwen 2.5 72B, Context Length=500 words2026.01 | 52.7 | — | — | — | — | — | — | |
| C-FedRAGSetting=Federated2026.05 | 50.6 | — | — | — | 47.8 | 5.3 | 2.97 | |
| Vanilla RAGBackbone=Llama 3.3 70B, Context Length=500 words2026.01 | 49.8 | — | — | — | — | — | — | |
| Vanilla RAGBackbone=Qwen 2.5 72B, Context Length=500 words2026.01 | 47.7 | — | — | — | — | — | — | |
| EfficientRAGSetting=Local2026.05 | 46.3 | — | — | — | 44.2 | 3.9 | 3.22 | |
| Vanilla RAGSetting=Local2026.05 | 43.5 | — | — | — | 45.2 | 2.6 | 1 | |
| LOCAL-RAGSetting=Federated2026.05 | 41.3 | — | — | — | 43.3 | 2.2 | 1 | |
| IterDRAGSetting=Local2026.05 | 38.8 | — | — | — | 43.8 | 6.37 | 3.02 | |
| Self-RAG + QREAM-ICLRAG Framework=Self-RAG Integration2026.04 | 36.8 | — | — | — | 30.5 | — | — | |
| QREAM-ICLRAG Framework=Standard RAG, Backbone LLM=Mistral-7B-Instruct2026.04 | 36.7 | — | — | — | 34.3 | — | — | |
| Self-RAG + QREAM-FTRAG Framework=Self-RAG Integration2026.04 | 36.2 | — | — | — | 31.4 | — | — | |
| QREAM-FTRAG Framework=Standard RAG, Backbone LLM=Mistral-7B-Instruct2026.04 | 35.9 | — | — | — | 33.6 | — | — | |
| QREAM-FTRAG Framework=Standard RAG, Backbone LLM=Llama-3-8B-Instruct2026.04 | 34.9 | — | — | — | 30.1 | — | — | |
| QREAM-ICLRAG Framework=Standard RAG, Backbone LLM=Llama-3-8B-Instruct2026.04 | 34.6 | — | — | — | 30.4 | — | — | |
| CurvPruneContext Budget (B)=20482026.02 | 13.1 | — | — | — | — | — | — | |
| BM25+TexContext Budget (B)=20482026.02 | 12.2 | — | — | — | — | — | — | |
| Head+TailContext Budget (B)=20482026.02 | 10.3 | — | — | — | — | — | — | |
| RandomContext Budget (B)=20482026.02 | 8.8 | — | — | — | — | — | — | |
| Sel. CtxContext Budget (B)=20482026.02 | 8.2 | — | — | — | — | — | — | |
| RecencyContext Budget (B)=20482026.02 | 6.6 | — | — | — | — | — | — | |
| LLMLinguaContext Budget (B)=20482026.02 | 6.3 | — | — | — | — | — | — | |
| BM25Context Budget (B)=20482026.02 | 3.4 | — | — | — | — | — | — | |
| Activation BeaconCompression Rate=2x, Backbone=Qwen2-7B-Instruct2026.02 | — | 45.75 | — | — | — | — | — | |
| Activation BeaconCompression Rate=4x, Backbone=Qwen2-7B-Instruct2026.02 | — | 45.29 | — | — | — | — | — | |
| Activation BeaconCompression Rate=8x, Backbone=Qwen2-7B-Instruct2026.02 | — | 45.12 | — | — | — | — | — | |
| Activation BeaconCompression Rate=16x, Backbone=Qwen2-7B-Instruct2026.02 | — | 36.59 | — | — | — | — | — | |
| Activation BeaconCompression Rate=32x, Backbone=Qwen2-7B-Instruct2026.02 | — | 39 | — | — | — | — | — | |
| BaselineBackbone=Llama3.1-8B2026.03 | — | — | — | — | 17.4 | — | — | |
| Block-AttnAttention Mechanism=Block-Attention2026.06 | — | 72.2 | — | — | — | — | — | |
| Block-AttnAttention Mechanism=Block-Attention, Inference Framework=vLLM2026.06 | — | 71.4 | — | — | — | — | — | |
| CacheBlendAttention Mechanism=CacheBlend2026.06 | — | 71.1 | — | — | — | — | — | |
| CoABackbone=Qwen2.5-0.5B, Context Window=8K2026.01 | — | — | 10.93 | — | — | — | — | |
| CoABackbone=Qwen2.5-1.5B, Context Window=8K2026.01 | — | — | 29.63 | — | — | — | — | |
| CoABackbone=Qwen2.5-7B, Context Window=8K2026.01 | — | — | 22.52 | — | — | — | — | |
| CoABackbone=Qwen2.5-14B, Context Window=8K2026.01 | — | — | 13.98 | — | — | — | — | |
| CoABackbone=Qwen2.5-32B, Context Window=8K2026.01 | — | — | 19.86 | — | — | — | — | |
| CoABackbone=Llama-3.2-3B-Instruct, Context Window=8K2026.01 | — | — | 29.66 | — | — | — | — | |
| CoABackbone=Llama-3.1-8B-Instruct, Context Window=8K2026.01 | — | — | 36.6 | — | — | — | — | |
| CoABackbone=Phi-3.5-mini-instruct, Context Window=8K2026.01 | — | — | 7.72 | — | — | — | — | |
| CoABackbone=Phi-3-small-8k-instruct, Context Window=8K2026.01 | — | — | 45.71 | — | — | — | — | |
| COMICompression Rate=2x, Backbone=Qwen2-7B-Instruct2026.02 | — | 61.8 | — | — | — | — | — | |
| COMICompression Rate=4x, Backbone=Qwen2-7B-Instruct2026.02 | — | 60.73 | — | — | — | — | — | |
| COMICompression Rate=8x, Backbone=Qwen2-7B-Instruct2026.02 | — | 57.71 | — | — | — | — | — | |
| COMICompression Rate=16x, Backbone=Qwen2-7B-Instruct2026.02 | — | 52.85 | — | — | — | — | — | |
| COMICompression Rate=32x, Backbone=Qwen2-7B-Instruct2026.02 | — | 47.56 | — | — | — | — | — | |
| CoTBackbone=GPT-4o2026.03 | — | — | — | — | 58.4 | — | — | |
| DeepNoteBackbone=Qwen3-32B2026.01 | — | — | — | 47.2 | — | — | — | |
| DeepNoteBackbone=Llama3.1-70B-Instruct2026.01 | — | — | — | 48.9 | — | — | — | |
| Full-AttnAttention Mechanism=Full-Attention2026.06 | — | 73.6 | — | — | — | — | — | |
| GenGroundBackbone=7B-class2026.03 | — | — | — | — | 45.6 | — | — | |
| IRCoTBackbone=Qwen3-32B2026.01 | — | — | — | 45.5 | — | — | — | |
| IRCoTBackbone=Llama3.1-70B-Instruct2026.01 | — | — | — | 37 | — | — | — | |
| IRCoTBackbone=GPT-4o2026.03 | — | — | — | — | 78 | — | — | |
| IRCoTBackbone=Llama3.1-8B2026.03 | — | — | — | — | 56.8 | — | — | |
| Iter-RetGenBackbone=Qwen3-32B2026.01 | — | — | — | 40.2 | — | — | — | |
| Iter-RetGenBackbone=Llama3.1-70B-Instruct2026.01 | — | — | — | 34.5 | — | — | — |