Information Retrieval on BEIR v1 (test)
49.2ArguAnacpt-text
Evaluation results
| Method | Links | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| cpt-textSetting=Transfer from MSMARCO, Zero-shot=true, Size=M2022.01 | 49.2 | — | 19.9 | 38.6 | 77 | 42.2 | 63.1 | 36.7 | — | 69.7 | — | 70.4 | 58.5 | 29.7 | 50.5 | — | |
| DistilSPLADE-maxpooling=max, distillation=true2021.09 | 47.9 | 43.3 | 23.5 | 43.5 | 78.6 | 33.6 | 68.4 | 33.4 | 52.1 | 83.8 | 15.8 | 69.3 | 71 | 36.4 | 50 | 50.6 | |
| Splade v2Setting=Transfer from MSMARCO, Zero-shot=true2022.01 | 47.9 | — | 23.5 | 43.5 | 78.6 | 33.6 | 68.4 | 33.4 | — | 83.8 | — | 69.3 | 71 | 27.2 | 52.7 | — | |
| cpt-textSetting=Transfer from MSMARCO, Zero-shot=true, Size=S2022.01 | 47 | — | 18.5 | 36.2 | 72.1 | 38.4 | 59.4 | 33.2 | — | 70.6 | — | 67.2 | 67.9 | 28.5 | 49 | — | |
| cpt-textSetting=Transfer from MSMARCO, Zero-shot=true, Size=L2022.01 | 46.9 | — | 19.4 | 41.2 | 75.6 | 45.2 | 64.8 | 38 | — | 67.7 | — | 74.4 | 56.2 | 30.9 | 50.9 | — | |
| ColBERT v2Setting=Transfer from MSMARCO, Zero-shot=true2022.01 | 46.3 | — | 17.6 | 44.6 | 78.5 | 35.6 | 66.7 | 33.8 | — | 85.2 | — | 69.3 | 73.8 | 26.3 | 52.5 | — | |
| SPLADE sumpooling=sum2021.09 | 44.7 | 38.7 | 16.2 | 34.3 | 72.8 | 25.8 | 63.5 | 31.1 | 43.8 | 82.9 | 14.1 | 62.6 | 65.5 | 28.9 | 44.6 | 45.1 | |
| ContrieverSetting=Transfer from MSMARCO, Zero-shot=true2022.01 | 44.6 | — | 23.7 | 41.3 | 75.8 | 32.9 | 63.8 | 32.8 | — | 86.5 | — | 67.7 | 59.6 | 23 | 50.2 | — | |
| SPLADE-maxpooling=max2021.09 | 43.9 | 40.2 | 19.9 | 36.6 | 73 | 28.7 | 63.6 | 31.3 | 46.9 | 83.5 | 14.5 | 62.8 | 67.3 | 31.6 | 46 | 46.4 | |
| cpt-textSetting=Transfer from MSMARCO, Zero-shot=true, Size=XL2022.01 | 43.5 | — | 22.3 | 43.2 | 77.5 | 51.2 | 68.8 | 40.7 | — | 63.8 | — | 75.4 | 64.9 | 29.1 | 52.8 | — | |
| TAS-BSetting=Transfer from MSMARCO, Zero-shot=true2022.01 | 42.9 | — | 22.8 | 38.4 | 70 | 30 | 58.4 | 31.9 | — | 83.5 | — | 64.3 | 48.1 | 16.2 | 46 | — | |
| TAS-BCategory=Baseline2021.09 | 42.7 | 40.8 | 22.8 | 38.4 | 70 | 30 | 58.4 | 31.9 | 46.3 | 83.5 | 14.9 | 64.3 | 48.1 | 17.3 | 43.5 | 43.7 | |
| ANCESetting=Transfer from MSMARCO, Zero-shot=true2022.01 | 41.5 | — | 19.8 | 28.1 | 66.9 | 29.5 | 45.6 | 23.7 | — | 85.2 | — | 50.7 | 65.4 | 24 | 43.7 | — | |
| cpt-textSetting=Unsupervised, Zero-shot=true, Size=M2022.01 | 41.2 | — | 15.6 | 29.6 | 58.2 | 37.3 | 53 | 34.5 | — | 70.3 | — | 68.3 | 44.3 | 23.3 | 43.2 | — | |
| cpt-textSetting=Unsupervised, Zero-shot=true, Size=L2022.01 | 39.2 | — | 16.1 | 31.2 | 63.8 | 39.7 | 54.3 | 36.9 | — | 68.7 | — | 71.2 | 42.7 | 22.8 | 44.2 | — | |
| cpt-textSetting=Unsupervised, Zero-shot=true, Size=S2022.01 | 38.7 | — | 15.8 | 27.2 | 57.1 | 34.1 | 51.5 | 32 | — | 68.1 | — | 65.4 | 52.9 | 21 | 42.2 | — | |
| ContrieverSetting=Unsupervised, Zero-shot=true2022.01 | 37.9 | — | 15.5 | 29.2 | 68.2 | 24.5 | 48.1 | 31.7 | — | 83.5 | — | 64.9 | 27.4 | 19.3 | 40.9 | — | |
| docT5querySetting=Transfer from MSMARCO, Zero-shot=true2022.01 | 34.9 | — | 20.1 | 33.1 | 71.4 | 29.1 | 58 | 32.8 | — | 80.2 | — | 67.5 | 71.3 | 34.7 | 48.5 | — | |
| BM25Category=Baseline2021.09 | 31.5 | 22.8 | 21.3 | 27.3 | 75.3 | 23.6 | 60.3 | 32.5 | 32.9 | 78.9 | 15.8 | 66.5 | 65.6 | 61.4 | 44 | 45.6 | |
| BM25Setting=Unsupervised, Zero-shot=true2022.01 | 31.5 | — | 21.3 | 31.3 | 75.3 | 23.6 | 60.3 | 32.5 | — | 78.9 | — | 66.5 | 65.6 | 36.7 | 47.6 | — | |
| BM25+CESetting=Transfer from MSMARCO, Zero-shot=true2022.01 | 31.1 | — | 25.3 | 39.2 | 81.9 | 34.7 | 70.7 | 35 | — | 82.5 | — | 68.8 | 75.7 | 27.1 | 52 | — | |
| ColBERTCategory=Baseline2021.09 | 23.3 | 42.5 | 18.4 | 39.2 | 77.1 | 31.7 | 59.3 | 30.5 | 52.4 | 85.4 | 14.5 | 67.1 | 67.7 | 27.5 | 45.5 | 45.7 |