Question Answering on MuSiQue (F1)
52.27F1 ScorePANINI
Evaluation results
| Method | Links | |
|---|---|---|
| PANINIRetrieval Category=Proposed Method2026.02 | 52.27 | |
| OracleBackbone=Llama 3.3 70B, Context Length=1500 words2026.01 | 51.9 | |
| OracleBackbone=Llama 3.3 70B, Context Length=1000 words2026.01 | 49.7 | |
| HippoRAG 2Retrieval Category=Structure-Augmented RAG2026.02 | 49.3 | |
| OracleBackbone=Qwen 2.5 72B, Context Length=1000 words2026.01 | 48.7 | |
| OracleBackbone=Qwen 2.5 72B, Context Length=1500 words2026.01 | 48.6 | |
| OracleBackbone=Llama 3.3 70B, Context Length=500 words2026.01 | 46.2 | |
| NV-Embed-v2 (7B)Retrieval Category=Dense Retrieval2026.02 | 46 | |
| IRCoTRetrieval Category=Agentic Systems2026.02 | 44.2 | |
| OracleBackbone=Qwen 2.5 72B, Context Length=500 words2026.01 | 43.8 | |
| Qwen3-Embedding (8B)Retrieval Category=Dense Retrieval, Reranker=Qwen3-Embedding (8B) + reranker2026.02 | 43.7 | |
| CoABackbone=text-unicorn, Context Window=8k2024.06 | 42.49 | |
| GraphRAGRetrieval Category=Structure-Augmented RAG2026.02 | 42 | |
| Search-R1Retrieval Category=Agentic Systems2026.02 | 41.1 | |
| RAGBackbone=text-unicorn, Context Window=8k2024.06 | 40.38 | |
| Qwen3-Embedding (8B)Retrieval Category=Dense Retrieval, Reranker=None2026.02 | 39.4 | |
| RAPTORRetrieval Category=Structure-Augmented RAG2026.02 | 39.2 | |
| DF-RAG + ICBackbone=Llama 3.3 70B, Context Length=1000 words2026.01 | 38.6 | |
| LongRAGBackbone=Llama 3.3 70B, Context Length=independent2026.01 | 38.1 | |
| DF-RAG + ICBackbone=Llama 3.3 70B, Context Length=1500 words2026.01 | 38 | |
| DF-RAGBackbone=Llama 3.3 70B, Context Length=1500 words2026.01 | 37.9 | |
| RAG with gMMRBackbone=Llama 3.3 70B, Context Length=1000 words2026.01 | 37.5 | |
| CoABackbone=text-bison, Context Window=8k2024.06 | 37.09 | |
| LongRAGBackbone=Qwen 2.5 72B, Context Length=independent2026.01 | 36.8 | |
| DF-RAGBackbone=Llama 3.3 70B, Context Length=1000 words2026.01 | 36.3 | |
| Vanilla RAGBackbone=Llama 3.3 70B, Context Length=1000 words2026.01 | 36 | |
| RAG with gMMRBackbone=Llama 3.3 70B, Context Length=1500 words2026.01 | 35.9 | |
| HippoRAGRetrieval Category=Structure-Augmented RAG2026.02 | 35.9 | |
| Vanilla RAGBackbone=Llama 3.3 70B, Context Length=1500 words2026.01 | 35.4 | |
| CoABackbone=gemini-ultra, Context Window=8k2024.06 | 35.09 | |
| RAG with gMMRBackbone=Qwen 2.5 72B, Context Length=1500 words2026.01 | 34.6 | |
| RAPTORBackbone=Llama 3.3 70B, Context Length=independent2026.01 | 34.3 | |
| DF-RAG + ICBackbone=Llama 3.3 70B, Context Length=500 words2026.01 | 34 | |
| RAGBackbone=text-bison, Context Window=8k2024.06 | 33.83 | |
| Vanilla RAGBackbone=Qwen 2.5 72B, Context Length=1500 words2026.01 | 33.6 | |
| HippoRAGBackbone=Llama 3.3 70B, Context Length=independent2026.01 | 33.5 | |
| DF-RAGBackbone=Llama 3.3 70B, Context Length=500 words2026.01 | 33.3 | |
| RAG with gMMRBackbone=Qwen 2.5 72B, Context Length=1000 words2026.01 | 33.3 | |
| DF-RAG + ICBackbone=Qwen 2.5 72B, Context Length=1500 words2026.01 | 33.3 | |
| HippoRAGBackbone=Qwen 2.5 72B, Context Length=independent2026.01 | 32.8 | |
| DF-RAG + ICBackbone=Qwen 2.5 72B, Context Length=500 words2026.01 | 32.7 | |
| DF-RAGBackbone=Qwen 2.5 72B, Context Length=1500 words2026.01 | 32.2 | |
| Vanilla RAGBackbone=Qwen 2.5 72B, Context Length=1000 words2026.01 | 31.9 | |
| DF-RAG + ICBackbone=Qwen 2.5 72B, Context Length=1000 words2026.01 | 31.7 | |
| RAGBackbone=gemini-ultra, Context Window=8k2024.06 | 31.56 | |
| DF-RAGBackbone=Qwen 2.5 72B, Context Length=500 words2026.01 | 31.3 | |
| RAG with gMMRBackbone=Llama 3.3 70B, Context Length=500 words2026.01 | 30.9 | |
| Vanilla RAGBackbone=Llama 3.3 70B, Context Length=500 words2026.01 | 29.9 | |
| BM25Retrieval Category=Sparse Retrieval, Reranker=BM25 + reranker2026.02 | 29.9 | |
| VanillaBackbone=text-unicorn, Context Window=8k2024.06 | 29.67 | |
| RAPTORBackbone=Qwen 2.5 72B, Context Length=independent2026.01 | 29.6 | |
| DF-RAGBackbone=Qwen 2.5 72B, Context Length=1000 words2026.01 | 28.8 | |
| VanillaBackbone=gemini-ultra, Context Window=32k2024.06 | 27.93 | |
| RAG with gMMRBackbone=Qwen 2.5 72B, Context Length=500 words2026.01 | 27.6 | |
| VanillaBackbone=text-bison, Context Window=8k2024.06 | 26.87 | |
| TIPSBackbone=Qwen-2.5-7B Instruct2026.03 | 26.58 | |
| BM25Retrieval Category=Sparse Retrieval, Reranker=None2026.02 | 25.3 | |
| Vanilla RAGBackbone=Qwen 2.5 72B, Context Length=500 words2026.01 | 24.8 | |
| VanillaBackbone=gemini-ultra, Context Window=8k2024.06 | 23.61 | |
| INTRAGenerator=T5Gemma2026.05 | 23 | |
| Qwen3-Emb-4B + RerankerGenerator=T5Gemma2026.05 | 22.5 | |
| NoneRetrieval Category=No Retrieval2026.02 | 22 | |
| Qwen3-Emb-4BGenerator=T5Gemma2026.05 | 21.8 | |
| Hybrid RAGGenerator=T5Gemma2026.05 | 20.1 | |
| Qwen3-Emb-0.6BGenerator=T5Gemma2026.05 | 20 | |
| MaxSimGenerator=T5Gemma2026.05 | 19.8 | |
| BGEGenerator=T5Gemma2026.05 | 19.7 | |
| PPOBackbone=Qwen-2.5-7B Instruct2026.03 | 17.79 | |
| BM25Generator=T5Gemma2026.05 | 16.8 | |
| TIPSBackbone=Qwen-2.5-3B Instruct2026.03 | 15.85 | |
| GRPOBackbone=Qwen-2.5-3B Instruct2026.03 | 14.81 | |
| TF-IDFGenerator=T5Gemma2026.05 | 14.6 | |
| MT-GRPO*Backbone=Qwen-2.5-7B Instruct2026.03 | 14.29 | |
| PPOBackbone=Qwen-2.5-3B Instruct2026.03 | 12.65 | |
| MT-PPOBackbone=Qwen-2.5-7B Instruct2026.03 | 12.13 | |
| GRPOBackbone=Qwen-2.5-7B Instruct2026.03 | 12.12 | |
| MT-PPOBackbone=Qwen-2.5-3B Instruct2026.03 | 11.05 | |
| MT-GRPO*Backbone=Qwen-2.5-3B Instruct2026.03 | 10.68 | |
| LightRAGRetrieval Category=Structure-Augmented RAG2026.02 | 9.3 |