Information Retrieval on Natural Questions
100Recall@1BLOCKSEARCH attn., R^any_19
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| BLOCKSEARCH attn., R^any_19Corpus Size (N)=0.5k2026.07 | 100 | — | — | — | — | — | — | — | |
| BLOCKSEARCH attn., R^any_19Corpus Size (N)=1k2026.07 | 100 | — | — | — | — | — | — | — | |
| BLOCKSEARCH attn., R^any_19Corpus Size (N)=5k2026.07 | 100 | — | — | — | — | — | — | — | |
| BLOCKSEARCH attn., R^any_19Corpus Size (N)=8.6k2026.07 | 97 | — | — | — | — | — | — | — | |
| BLOCKSEARCH attn., R^any_19Corpus Size (N)=2.5k2026.07 | 96.2 | — | — | — | — | — | — | — | |
| Qwen3-denseCorpus Size (N)=0.5k2026.07 | 95.5 | — | — | — | — | — | — | — | |
| BLOCKSEARCH-routingCorpus Size (N)=0.5k, Top-B=256, Routing Layer=L162026.07 | 93.2 | — | — | — | — | — | — | — | |
| BLOCKSEARCHCorpus Size (N)=0.5k2026.07 | 92.2 | — | — | — | — | — | — | — | |
| BLOCKSEARCH-SSMax+routingCorpus Size (N)=0.5k2026.07 | 91.5 | — | — | — | — | — | — | — | |
| BLOCKSEARCH-sinkCorpus Size (N)=0.5k2026.07 | 91.2 | — | — | — | — | — | — | — | |
| BLOCKSEARCH-SSMaxCorpus Size (N)=0.5k, Attention Layers=282026.07 | 90.5 | — | — | — | — | — | — | — | |
| Qwen3-denseCorpus Size (N)=1k2026.07 | 86.5 | — | — | — | — | — | — | — | |
| BLOCKSEARCH-routingCorpus Size (N)=1k, Top-B=256, Routing Layer=L162026.07 | 82.7 | — | — | — | — | — | — | — | |
| BLOCKSEARCH-SSMax+routingCorpus Size (N)=1k2026.07 | 81.4 | — | — | — | — | — | — | — | |
| BLOCKSEARCH-SSMaxCorpus Size (N)=1k, Attention Layers=282026.07 | 79.5 | — | — | — | — | — | — | — | |
| BLOCKSEARCHCorpus Size (N)=1k2026.07 | 77.7 | — | — | — | — | — | — | — | |
| BLOCKSEARCH-sinkCorpus Size (N)=1k2026.07 | 76.2 | — | — | — | — | — | — | — | |
| BLOCKSEARCH-routingCorpus Size (N)=2.5k, Top-B=256, Routing Layer=L162026.07 | 63.2 | — | — | — | — | — | — | — | |
| Qwen3-denseCorpus Size (N)=2.5k2026.07 | 62.9 | — | — | — | — | — | — | — | |
| BLOCKSEARCH-SSMax+routingCorpus Size (N)=2.5k2026.07 | 61.4 | — | — | — | — | — | — | — | |
| MSA-4BCorpus Size (N)=0.5k2026.07 | 59.7 | — | — | — | — | — | — | — | |
| BLOCKSEARCH-SSMaxCorpus Size (N)=2.5k, Attention Layers=282026.07 | 58.1 | — | — | — | — | — | — | — | |
| MSA-4BCorpus Size (N)=1k2026.07 | 51.6 | — | — | — | — | — | — | — | |
| Qwen3-denseCorpus Size (N)=5k2026.07 | 51.4 | — | — | — | — | — | — | — | |
| BLOCKSEARCH-routingCorpus Size (N)=5k, Top-B=256, Routing Layer=L162026.07 | 48.1 | — | — | — | — | — | — | — | |
| BLOCKSEARCH-SSMax+routingCorpus Size (N)=5k2026.07 | 46.1 | — | — | — | — | — | — | — | |
| BLOCKSEARCH-sinkCorpus Size (N)=2.5k2026.07 | 45.6 | — | — | — | — | — | — | — | |
| BLOCKSEARCHCorpus Size (N)=2.5k2026.07 | 43.9 | — | — | — | — | — | — | — | |
| BLOCKSEARCH-SSMaxCorpus Size (N)=5k, Attention Layers=282026.07 | 42.6 | — | — | — | — | — | — | — | |
| Qwen3-denseCorpus Size (N)=8.6k2026.07 | 39.6 | — | — | — | — | — | — | — | |
| MSA-4BCorpus Size (N)=2.5k2026.07 | 36.3 | — | — | — | — | — | — | — | |
| BLOCKSEARCH-routingCorpus Size (N)=8.6k, Top-B=256, Routing Layer=L162026.07 | 34.6 | — | — | — | — | — | — | — | |
| BLOCKSEARCH-SSMax+routingCorpus Size (N)=8.6k2026.07 | 34.3 | — | — | — | — | — | — | — | |
| BLOCKSEARCH-SSMaxCorpus Size (N)=8.6k, Attention Layers=282026.07 | 30.3 | — | — | — | — | — | — | — | |
| MSA-4BCorpus Size (N)=5k2026.07 | 26.8 | — | — | — | — | — | — | — | |
| MSA-4BCorpus Size (N)=8.6k2026.07 | 18.6 | — | — | — | — | — | — | — | |
| BLOCKSEARCH-sinkCorpus Size (N)=5k2026.07 | 10.3 | — | — | — | — | — | — | — | |
| BLOCKSEARCHCorpus Size (N)=5k2026.07 | 4.8 | — | — | — | — | — | — | — | |
| BLOCKSEARCH-sinkCorpus Size (N)=8.6k2026.07 | 1.5 | — | — | — | — | — | — | — | |
| BLOCKSEARCHCorpus Size (N)=8.6k2026.07 | 0.2 | — | — | — | — | — | — | — | |
| BGE2026.01 | — | 73 | 90 | 69 | — | — | — | — | |
| BGEBackbone=BGE2025.04 | — | — | — | — | 35.3 | 80.2 | 89.8 | — | |
| Cawai-BGEBackbone=BGE2025.04 | — | — | — | — | 31.6 | 78.8 | 88.8 | — | |
| Cawai-DPRBackbone=DPR2025.04 | — | — | — | — | 33.9 | 74.8 | 86 | — | |
| Cawai-GTRBackbone=GTR2025.04 | — | — | — | — | 37.5 | 79.2 | 88.5 | — | |
| DIMEPruning Ratio=20%2026.01 | — | 72 | 89 | 68 | — | — | — | — | |
| DIMEPruning Ratio=40%2026.01 | — | 72 | 90 | 67 | — | — | — | — | |
| DIMEPruning Ratio=60%2026.01 | — | 70 | 88 | 66 | — | — | — | — | |
| DIMEPruning Ratio=80%2026.01 | — | 64 | 83 | 60 | — | — | — | — | |
| DPRBackbone=DPR2025.04 | — | — | — | — | 30.6 | 75.2 | 86.3 | — | |
| e5-large-v22026.01 | — | 72 | 90 | 68 | — | — | — | — | |
| GTRBackbone=GTR2025.04 | — | — | — | — | 41.4 | 81.9 | 90.2 | — | |
| HybridBackbone=DPR, Mode=Hybrid2025.04 | — | — | — | — | 37 | 78.3 | 87.8 | — | |
| HybridBackbone=GTR, Mode=Hybrid2025.04 | — | — | — | — | 41.4 | 82 | 90.3 | — | |
| HybridBackbone=BGE, Mode=Hybrid2025.04 | — | — | — | — | 35.3 | 81.3 | 90.4 | — | |
| HybridRetriever=BERT2026.06 | — | — | — | — | — | — | — | 70.11 | |
| HybridRetriever=BGE-M32026.06 | — | — | — | — | — | — | — | 82.41 | |
| HybridRetriever=Qwen3-Embedding-4B2026.06 | — | — | — | — | — | — | — | 81.14 | |
| HYPENCODERLayers=22026.01 | — | 43 | 60 | 40 | — | — | — | — | |
| HYPENCODERLayers=42026.01 | — | 44 | 60 | 40 | — | — | — | — | |
| HYPENCODERLayers=62026.01 | — | 44 | 61 | 40 | — | — | — | — | |
| HYPENCODERLayers=82026.01 | — | 45 | 61 | 42 | — | — | — | — | |
| IMRNNsBackbone=e5-large-v22026.01 | — | 75 | 93 | 71 | — | — | — | — | |
| IMRNNsBackbone=MiniLM2026.01 | — | 91 | 69 | 75 | — | — | — | — | |
| IMRNNsBackbone=BGE2026.01 | — | 75 | 93 | 71 | — | — | — | — | |
| LongLLMLinguaRetriever=BERT2026.06 | — | — | — | — | — | — | — | 44.77 | |
| LongLLMLinguaRetriever=BGE-M32026.06 | — | — | — | — | — | — | — | 85.62 | |
| LongLLMLinguaRetriever=Qwen3-Embedding-4B2026.06 | — | — | — | — | — | — | — | 86.01 | |
| LongRAGRetriever=BERT2026.06 | — | — | — | — | — | — | — | 44.41 | |
| LongRAGRetriever=BGE-M32026.06 | — | — | — | — | — | — | — | 91.01 | |
| LongRAGRetriever=Qwen3-Embedding-4B2026.06 | — | — | — | — | — | — | — | 92.53 | |
| MiniLM2026.01 | — | 89 | 68 | 73 | — | — | — | — | |
| MS-PoERetriever=BERT2026.06 | — | — | — | — | — | — | — | 44.77 | |
| MS-PoERetriever=BGE-M32026.06 | — | — | — | — | — | — | — | 85.62 | |
| MS-PoERetriever=Qwen3-Embedding-4B2026.06 | — | — | — | — | — | — | — | 86.01 | |
| PINERetriever=BERT2026.06 | — | — | — | — | — | — | — | 44.77 | |
| PINERetriever=BGE-M32026.06 | — | — | — | — | — | — | — | 85.62 | |
| PINERetriever=Qwen3-Embedding-4B2026.06 | — | — | — | — | — | — | — | 86.01 | |
| RAGRetriever=SPLADEv32026.06 | — | — | — | — | — | — | — | 79.94 | |
| RAGRetriever=BERT2026.06 | — | — | — | — | — | — | — | 44.77 | |
| RAGRetriever=BGE-M32026.06 | — | — | — | — | — | — | — | 85.62 | |
| RAGRetriever=Qwen3-Embedding-4B2026.06 | — | — | — | — | — | — | — | 86.01 | |
| REPLUGRetriever=BERT2026.06 | — | — | — | — | — | — | — | 44.77 | |
| REPLUGRetriever=BGE-M32026.06 | — | — | — | — | — | — | — | 85.62 | |
| REPLUGRetriever=Qwen3-Embedding-4B2026.06 | — | — | — | — | — | — | — | 86.01 | |
| SearchAd2026.01 | — | 71 | 89 | 67 | — | — | — | — | |
| Summarized RAGRetriever=BERT2026.06 | — | — | — | — | — | — | — | 44.77 | |
| Summarized RAGRetriever=BGE-M32026.06 | — | — | — | — | — | — | — | 85.62 | |
| Summarized RAGRetriever=Qwen3-Embedding-4B2026.06 | — | — | — | — | — | — | — | 86.01 | |
| UMG-RAGRetriever=BERT2026.06 | — | — | — | — | — | — | — | 68.52 | |
| UMG-RAGRetriever=BGE-M32026.06 | — | — | — | — | — | — | — | 80.23 | |
| UMG-RAGRetriever=Qwen3-Embedding-4B2026.06 | — | — | — | — | — | — | — | 79.49 | |
| UMGP-RAGRetriever=BERT2026.06 | — | — | — | — | — | — | — | 77.97 | |
| UMGP-RAGRetriever=BGE-M32026.06 | — | — | — | — | — | — | — | 87.59 | |
| UMGP-RAGRetriever=Qwen3-Embedding-4B2026.06 | — | — | — | — | — | — | — | 87.45 |