Question Answering on MultifieldQA
57.2F1 ScoreOracle
Evaluation results
| Method | Links | ||
|---|---|---|---|
| OracleBackbone=Qwen 2.5 72B, Context Length=500 words2026.01 | 57.2 | — | |
| OracleBackbone=Qwen 2.5 72B, Context Length=1000 words2026.01 | 56.7 | — | |
| OracleBackbone=Llama 3.3 70B, Context Length=1000 words2026.01 | 56.2 | — | |
| OracleBackbone=Llama 3.3 70B, Context Length=1500 words2026.01 | 56.1 | — | |
| OracleBackbone=Qwen 2.5 72B, Context Length=1500 words2026.01 | 55.9 | — | |
| OracleBackbone=Llama 3.3 70B, Context Length=500 words2026.01 | 54.9 | — | |
| RAG with AttentionRetriever-LlamaLLM Backbone=Llama-3.1 8B, Retrieval Method=AttentionRetriever-Llama2026.02 | 54.36 | 416.46 | |
| BaselineLLM Backbone=Llama-3.1 8B, Context Type=Full Context2026.02 | 54.3 | 6,996.87 | |
| RAG with AttentionRetriever-QwenLLM Backbone=Llama-3.1 8B, Retrieval Method=AttentionRetriever-Qwen2026.02 | 54.13 | 432.38 | |
| LongRAGBackbone=Llama 3.3 70B, Context Length=independent2026.01 | 52.9 | — | |
| RAG with AttentionRetriever-QwenLLM Backbone=Qwen-2.5 7B, Retrieval Method=AttentionRetriever-Qwen2026.02 | 52.74 | 416.63 | |
| RAG with gMMRBackbone=Qwen 2.5 72B, Context Length=1000 words2026.01 | 51.9 | — | |
| RAG with SPScannerLLM Backbone=Llama-3.1 8B, Retrieval Method=SPScanner2026.02 | 51.73 | 449.22 | |
| DF-RAGBackbone=Llama 3.3 70B, Context Length=1000 words2026.01 | 51.6 | — | |
| DF-RAGBackbone=Qwen 2.5 72B, Context Length=500 words2026.01 | 51 | — | |
| RAG with gMMRBackbone=Llama 3.3 70B, Context Length=1000 words2026.01 | 50.9 | — | |
| RAG with gMMRBackbone=Llama 3.3 70B, Context Length=1500 words2026.01 | 50.9 | — | |
| BaselineLLM Backbone=Qwen-2.5 7B, Context Type=Full Context2026.02 | 50.85 | 7,205.38 | |
| RAG with AttentionRetriever-LlamaLLM Backbone=Qwen-2.5 7B, Retrieval Method=AttentionRetriever-Llama2026.02 | 50.62 | 400.66 | |
| Vanilla RAGBackbone=Qwen 2.5 72B, Context Length=1000 words2026.01 | 50.6 | — | |
| DF-RAGBackbone=Llama 3.3 70B, Context Length=1500 words2026.01 | 50.6 | — | |
| DF-RAGBackbone=Qwen 2.5 72B, Context Length=1000 words2026.01 | 50.4 | — | |
| Vanilla RAGBackbone=Llama 3.3 70B, Context Length=1500 words2026.01 | 50.4 | — | |
| RAG with gMMRBackbone=Qwen 2.5 72B, Context Length=1500 words2026.01 | 50.1 | — | |
| Vanilla RAGBackbone=Llama 3.3 70B, Context Length=1000 words2026.01 | 50 | — | |
| DF-RAGBackbone=Qwen 2.5 72B, Context Length=1500 words2026.01 | 50 | — | |
| RAG with gMMRBackbone=Qwen 2.5 72B, Context Length=500 words2026.01 | 49.7 | — | |
| RAPTORBackbone=Qwen 2.5 72B, Context Length=independent2026.01 | 49.6 | — | |
| RAG with AttentionRetriever-QwenLLM Backbone=GPT-4o mini, Retrieval Method=AttentionRetriever-Qwen2026.02 | 49.47 | 393.35 | |
| BaselineLLM Backbone=Mistral-7B v0.3, Context Type=Full Context2026.02 | 49.42 | 7,931.67 | |
| DF-RAG + ICBackbone=Qwen 2.5 72B, Context Length=1000 words2026.01 | 49.3 | — | |
| RAG with AttentionRetriever-QwenLLM Backbone=Mistral-7B v0.3, Retrieval Method=AttentionRetriever-Qwen2026.02 | 49.04 | 451.57 | |
| HippoRAGBackbone=Qwen 2.5 72B, Context Length=independent2026.01 | 48.9 | — | |
| DF-RAG + ICBackbone=Qwen 2.5 72B, Context Length=500 words2026.01 | 48.8 | — | |
| DF-RAG + ICBackbone=Qwen 2.5 72B, Context Length=1500 words2026.01 | 48.7 | — | |
| DF-RAG + ICBackbone=Llama 3.3 70B, Context Length=1000 words2026.01 | 48.6 | — | |
| RAG with AttentionRetriever-LlamaLLM Backbone=GPT-4o mini, Retrieval Method=AttentionRetriever-Llama2026.02 | 48.55 | 377.87 | |
| RAG with AttentionRetriever-LlamaLLM Backbone=Mistral-7B v0.3, Retrieval Method=AttentionRetriever-Llama2026.02 | 48.54 | 431.09 | |
| DF-RAGBackbone=Llama 3.3 70B, Context Length=500 words2026.01 | 48.3 | — | |
| RAPTORBackbone=Llama 3.3 70B, Context Length=independent2026.01 | 48.3 | — | |
| HippoRAGBackbone=Llama 3.3 70B, Context Length=independent2026.01 | 48.3 | — | |
| RAG with gMMRBackbone=Llama 3.3 70B, Context Length=500 words2026.01 | 48.2 | — | |
| Vanilla RAGBackbone=Qwen 2.5 72B, Context Length=1500 words2026.01 | 48.1 | — | |
| LongRAGBackbone=Qwen 2.5 72B, Context Length=independent2026.01 | 47.8 | — | |
| Vanilla RAGBackbone=Llama 3.3 70B, Context Length=500 words2026.01 | 47.7 | — | |
| RAG with SPScannerLLM Backbone=Qwen-2.5 7B, Retrieval Method=SPScanner2026.02 | 47.66 | 438.25 | |
| Vanilla RAGBackbone=Qwen 2.5 72B, Context Length=500 words2026.01 | 47.5 | — | |
| DF-RAG + ICBackbone=Llama 3.3 70B, Context Length=500 words2026.01 | 47 | — | |
| DF-RAG + ICBackbone=Llama 3.3 70B, Context Length=1500 words2026.01 | 46.5 | — | |
| RAG with SPScannerLLM Backbone=GPT-4o mini, Retrieval Method=SPScanner2026.02 | 44.6 | 409.96 | |
| BaselineLLM Backbone=GPT-4o mini, Context Type=Full Context2026.02 | 44.55 | 6,899.97 | |
| RAG with SPScannerLLM Backbone=Mistral-7B v0.3, Retrieval Method=SPScanner2026.02 | 44.02 | 470.87 |