Information Retrieval on Natural Questions (test)
92.53AR@5LongRAG
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| LongRAGRetriever=Qwen3-Embedding-4B2026.06 | 92.53 | — | — | — | — | |
| LongRAGRetriever=BGE-M32026.06 | 91.01 | — | — | — | — | |
| UMGP-RAGRetriever=BGE-M32026.06 | 87.59 | — | — | — | — | |
| UMGP-RAGRetriever=Qwen3-Embedding-4B2026.06 | 87.45 | — | — | — | — | |
| RAGRetriever=Qwen3-Embedding-4B2026.06 | 86.01 | — | — | — | — | |
| Summarized RAGRetriever=Qwen3-Embedding-4B2026.06 | 86.01 | — | — | — | — | |
| LongLLMLinguaRetriever=Qwen3-Embedding-4B2026.06 | 86.01 | — | — | — | — | |
| MS-PoERetriever=Qwen3-Embedding-4B2026.06 | 86.01 | — | — | — | — | |
| REPLUGRetriever=Qwen3-Embedding-4B2026.06 | 86.01 | — | — | — | — | |
| PINERetriever=Qwen3-Embedding-4B2026.06 | 86.01 | — | — | — | — | |
| RAGRetriever=BGE-M32026.06 | 85.62 | — | — | — | — | |
| Summarized RAGRetriever=BGE-M32026.06 | 85.62 | — | — | — | — | |
| LongLLMLinguaRetriever=BGE-M32026.06 | 85.62 | — | — | — | — | |
| MS-PoERetriever=BGE-M32026.06 | 85.62 | — | — | — | — | |
| REPLUGRetriever=BGE-M32026.06 | 85.62 | — | — | — | — | |
| PINERetriever=BGE-M32026.06 | 85.62 | — | — | — | — | |
| HybridRetriever=BGE-M32026.06 | 82.41 | — | — | — | — | |
| HybridRetriever=Qwen3-Embedding-4B2026.06 | 81.14 | — | — | — | — | |
| UMG-RAGRetriever=BGE-M32026.06 | 80.23 | — | — | — | — | |
| RAGRetriever=SPLADEv32026.06 | 79.94 | — | — | — | — | |
| UMG-RAGRetriever=Qwen3-Embedding-4B2026.06 | 79.49 | — | — | — | — | |
| UMGP-RAGRetriever=BERT2026.06 | 77.97 | — | — | — | — | |
| HybridRetriever=BERT2026.06 | 70.11 | — | — | — | — | |
| UMG-RAGRetriever=BERT2026.06 | 68.52 | — | — | — | — | |
| RAGRetriever=BERT2026.06 | 44.77 | — | — | — | — | |
| Summarized RAGRetriever=BERT2026.06 | 44.77 | — | — | — | — | |
| LongLLMLinguaRetriever=BERT2026.06 | 44.77 | — | — | — | — | |
| MS-PoERetriever=BERT2026.06 | 44.77 | — | — | — | — | |
| REPLUGRetriever=BERT2026.06 | 44.77 | — | — | — | — | |
| PINERetriever=BERT2026.06 | 44.77 | — | — | — | — | |
| LongRAGRetriever=BERT2026.06 | 44.41 | — | — | — | — | |
| ANCEsupervision level=supervised, #Params=220M2022.12 | — | — | 81.9 | — | 87.5 | |
| BM25supervision level=supervised, #Params=not specified2022.12 | — | 23.9 | 45.9 | 63.8 | 78.9 | |
| BM25Index-M (GB)=2.42022.10 | — | — | — | 63.9 | 78.8 | |
| BM25+DPRIndex-M (GB)=63.92022.10 | — | — | — | 82.6 | 88.6 | |
| BPRIndex-M (GB)=22022.10 | — | — | — | 77.9 | 85.7 | |
| coCondensersupervision level=supervised, #Params=220M2022.12 | — | — | 75.8 | 84.3 | 89 | |
| ColBERTsupervision level=supervised, #Params=110M2022.12 | — | — | — | 79.1 | — | |
| ColBERT-NQsupervision level=supervised, #Params=110M2022.12 | — | 54.3 | 75.7 | 85.6 | 90 | |
| DensePhrasesupervision level=supervised, #Params=330M2022.12 | — | 51.1 | 69.9 | 78.7 | — | |
| DPRsupervision level=supervised, #Params=220M2022.12 | — | 45.9 | 68.1 | 80 | 85.9 | |
| DPRIndex-M (GB)=61.52022.10 | — | — | — | 79.5 | 86.1 | |
| DPR_newsupervision level=supervised, #Params=220M2022.12 | — | 52.5 | 72.2 | 81.3 | 87.3 | |
| DPR-PAQsupervision level=supervised, #Params=220M2022.12 | — | — | 74.2 | 84 | 89.2 | |
| DPR32Index-M (GB)=2.52022.10 | — | — | — | 70.4 | 80 | |
| DrBoostIndex-M (GB)=15.42022.10 | — | — | — | 81.3 | 87.4 | |
| DrBoost-2Index-M (GB)=5.12022.10 | — | — | — | 77.3 | 84.5 | |
| FiD-KDsupervision level=semi/unsupervised, #Params=220M2022.12 | — | 49.4 | 73.8 | 84.3 | 89.3 | |
| Hybrid-DPR32Index-M (GB)=4.92022.10 | — | — | — | 77.7 | 86.2 | |
| Hybrid-DrBoost-2Index-M (GB)=7.52022.10 | — | — | — | 80.4 | 87.5 | |
| Hybrid-LITEIndex-M (GB)=4.92022.10 | — | — | — | 79.9 | 87.2 | |
| LITEIndex-M (GB)=2.52022.10 | — | — | — | 75.1 | 83.4 | |
| ReAtt BM25supervision level=semi/unsupervised, #Params=165M, retrieval source=BM252022.12 | — | 55.8 | 77.4 | 86 | 90.4 | |
| ReAtt DPRsupervision level=semi/unsupervised, #Params=165M, retrieval source=DPR2022.12 | — | 54.6 | 77.2 | 86.1 | 90.7 | |
| SPARIndex-M (GB)=1232022.10 | — | — | — | 83.6 | 88.8 | |
| YONO_w/ PTsupervision level=semi/unsupervised, #Params=165M, retrieval pretraining=true2022.12 | — | — | 75.3 | 85.2 | 90.2 | |
| YONO_w/o PTsupervision level=semi/unsupervised, #Params=165M, retrieval pretraining=false2022.12 | — | — | — | 72.3 | 82.2 |