Information Retrieval on BEIR v1.0 (test)
94.7FEVER ScoreGTR
Evaluation results
| Method | Links | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GTRRetriever Deployment=One Retriever for All, Model Scale=xxl2023.04 | 94.7 | 91.6 | 98.3 | 46.6 | 55.6 | 90 | 40.7 | 30 | 94.6 | 75.2 | 78 | 36.6 | 49.4 | 99.7 | 68.4 | — | |
| PTRretrieverRetriever Deployment=One Retriever per Domain2023.04 | 94.1 | — | 98.9 | 47.5 | 53.1 | 91.8 | 55.9 | 30.6 | 89.8 | 74.6 | 76.5 | 41.6 | 46.3 | 99.6 | 69.2 | — | |
| XTRRetriever Deployment=One Retriever for All, Model Scale=xxl2023.04 | 93.7 | 93 | 95.6 | 52.7 | 56.2 | 95 | 62.1 | 30.7 | 95.8 | 82.2 | 73 | 39.4 | 54.5 | 99.3 | 71.6 | — | |
| ColBERTRetriever Deployment=One Retriever for All2023.04 | 93.4 | 86.5 | 91.4 | 43.9 | 44.4 | 87.8 | 46.4 | 25.4 | 91.2 | 74.8 | 60.3 | 34.4 | 46.1 | 98.9 | 64.5 | — | |
| BM25Retriever Deployment=One Retriever for All2023.04 | 93.1 | 65.8 | 94.2 | 53.8 | 43.6 | 90.8 | 49.8 | 25 | 76 | 74 | 53.9 | 35.6 | 39.8 | 97.3 | 63.6 | — | |
| GenQRetriever Deployment=One Retriever per Domain2023.04 | 92.8 | 88.4 | 97.8 | 45.1 | 45 | 89.3 | 45.6 | 28 | 86.2 | 67.3 | 61.8 | 33.2 | 43.1 | 98.9 | 64.2 | — | |
| XTRRetriever Deployment=One Retriever for All, Model Scale=base2023.04 | 92.5 | 91 | 92.1 | 50.8 | 51.6 | 90.5 | 57.3 | 28 | 91.6 | 80.7 | 63.5 | 34.8 | 52 | 98.9 | 68 | — | |
| GTRRetriever Deployment=One Retriever for All, Model Scale=base2023.04 | 92.3 | 89.8 | 97.4 | 44.3 | 52.2 | 87.2 | 41.1 | 27.5 | 89.3 | 67.6 | 67 | 34 | 41.8 | 99.6 | 64.7 | — | |
| T5-ColBERTRetriever Deployment=One Retriever for All, Model Scale=xxl2023.04 | 91.7 | 93.3 | 81.4 | 50.1 | 49.8 | 94.6 | 60.3 | 29 | 95.5 | 81.6 | 72.5 | 38.5 | 54.6 | 99.1 | 69.1 | — | |
| T5-ColBERTRetriever Deployment=One Retriever for All, Model Scale=base2023.04 | 90.4 | 91.8 | 76 | 49.9 | 46.2 | 91.3 | 55.4 | 27.6 | 90.5 | 78.3 | 63 | 34.2 | 50.5 | 97.9 | 65.5 | — | |
| PLAID 1bitColBERT Model=colbert-small-v1, Compression=1 bit residual2026.06 | 89.5 | 71.4 | 31.4 | 36.7 | 83.5 | 73.4 | 25 | 35.3 | 58.1 | 74.6 | — | 17.9 | 44 | 86.8 | 54.8 | 39.8 | |
| PLAID 1bitColBERT Model=PLAID XLMR ET, Compression=1 bit residual2026.06 | 81.7 | 70.3 | 32.4 | 24.7 | 42.8 | 69.8 | 21.4 | 34.3 | 53.5 | 59 | — | 16.4 | 41.2 | 85 | 48.1 | 41 | |
| SPLADEv32026.06 | 79.6 | 44 | 50.9 | 29.3 | 74.8 | 71 | 23.3 | 35.7 | 58.6 | 69.2 | — | 15.8 | 45 | 81.4 | 51.1 | 37.4 | |
| ColBERTSaR + BM25ColBERT Model=colbert-small-v1, Ensemble=BM252026.06 | 74.1 | 61.2 | 34.3 | 56 | 71.7 | 73 | 17.7 | 35.5 | 43.9 | 67 | — | 16.9 | 36.6 | 84.4 | 50.5 | 33.8 | |
| ColBERTSaR w/ Official TrainColBERT Model=colbert-small-v1, Anchor Optimization Training Queries=Official Train2026.06 | 72.1 | 63.2 | — | — | — | 73.5 | — | 36.7 | — | 59.3 | — | — | — | — | — | 38.3 | |
| ColBERTSaR w/ MSMARCOColBERT Model=colbert-small-v1, Anchor Optimization Training Queries=MSMARCO2026.06 | 71.8 | 63.2 | 32.3 | 39.7 | 81 | 73.8 | 16 | 36.6 | 46.6 | 58.3 | — | 17.7 | 34.7 | 83.8 | 49.5 | 37.7 | |
| ColBERTSaRColBERT Model=colbert-small-v12026.06 | 71.6 | 61.8 | 32.2 | 38.9 | 77.1 | 73.7 | 15.2 | 36.2 | 46.2 | 58.9 | — | 17.6 | 34.5 | 84 | 49 | 37.6 | |
| ColBERTSaRColBERT Model=PLAID XLMR ET2026.06 | 67.4 | 56.5 | 33.9 | 32.6 | 55 | 67.1 | 14.7 | 34.7 | 45.6 | 48.1 | — | 16.4 | 33.4 | 79.8 | 44.5 | 37.6 | |
| BM252026.06 | 65.1 | 51.2 | 30 | 64.1 | 59.5 | 67.9 | 16.5 | 32.2 | 30.5 | 63.3 | — | 14.9 | 31.8 | 78.9 | 45 | 23.6 | |
| PLAID 0bitColBERT Model=colbert-small-v1, Compression=0 bit residual2026.06 | 63.1 | 41.6 | 29.9 | 35 | 78.2 | 69.3 | 15.5 | 31.5 | 32.2 | 48.6 | — | 16.4 | 22.5 | 76.9 | 42.4 | 33 |