Image-to-text retrieval on MSCOCO 5K (test)
84.8R@1BEIT-3
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| BEIT-3# Params=1.9B2022.11 | 84.8 | — | — | — | — | |
| X2-VLM_large# Params=593M2022.11 | 84.4 | — | — | — | — | |
| BLIPPre-train # Images=14M, fine-tuned=true2023.12 | 80.6 | 95.2 | 97.6 | — | — | |
| MAFA (4M-Clean)Pre-train # Images=4M, fine-tuned=true2023.12 | 79.4 | 94.4 | 97.5 | — | — | |
| MAFAPre-train # Images=4M, fine-tuned=true2023.12 | 78 | 94.1 | 97.2 | — | — | |
| ALBEFPre-train # Images=14M, fine-tuned=true2023.12 | 77.6 | 94.3 | 97.2 | — | — | |
| CEPT Pairs=15.2M, Architecture Type=Cross-Encoder, Evaluation Protocol=Fine-tuned2024.07 | 77.6 | 94.3 | 97.2 | 504.6 | — | |
| GRIT-VLP*Pre-train # Images=4M, fine-tuned=true2023.12 | 76.6 | 93.4 | 96.9 | — | — | |
| BLIP* (4M-Clean)Pre-train # Images=4M, fine-tuned=true2023.12 | 76.5 | 93.2 | 96.8 | — | — | |
| TCLPre-train # Images=4M, fine-tuned=true2023.12 | 75.6 | 92.8 | 96.7 | — | — | |
| VinVL-basePT Pairs=8.9M, Architecture Type=Cross-Encoder, Evaluation Protocol=Fine-tuned2024.07 | 74.6 | 92.6 | 96.3 | 494.9 | — | |
| CEPT Pairs=5.1M, Architecture Type=Cross-Encoder, Evaluation Protocol=Fine-tuned2024.07 | 74.4 | 92.3 | 96.5 | 493.7 | — | |
| ALBEFPre-train # Images=4M, fine-tuned=true2023.12 | 73.1 | 91.4 | 96 | — | — | |
| ALBEFPT Pairs=5.1M, Architecture Type=Cross-Encoder, Evaluation Protocol=Fine-tuned2024.07 | 73.1 | 91.4 | 96 | 488 | — | |
| COOKIEPT Pairs=5.9M, Architecture Type=Dual-Encoder, Evaluation Protocol=Fine-tuned, Ensemble=true, Visual Encoder Pre-training=true2024.07 | 71.6 | 90.9 | 95.4 | 481.6 | — | |
| CPRDPT Pairs=5.1M, Architecture Type=Dual-Encoder, Evaluation Protocol=Fine-tuned, Teacher Model PT Pairs=15.2M2024.07 | 70.8 | 91.7 | 96.2 | 481.2 | — | |
| OSCARPre-train # Images=4M, fine-tuned=true2023.12 | 70 | 91.1 | 95.5 | — | — | |
| Oscar-basePT Pairs=6.5M, Architecture Type=Cross-Encoder, Evaluation Protocol=Fine-tuned2024.07 | 70 | 91.1 | 95.5 | 479.9 | — | |
| CPRDPT Pairs=5.1M, Architecture Type=Dual-Encoder, Evaluation Protocol=Fine-tuned, Teacher Model PT Pairs=5.1M2024.07 | 69.7 | 91.1 | 95.8 | 474.4 | — | |
| COTSPT Pairs=15.3M, Architecture Type=Dual-Encoder, Evaluation Protocol=Fine-tuned2024.07 | 69 | 90.4 | 94.9 | 472.6 | — | |
| VSE∞Architecture Type=Dual-Encoder, Evaluation Protocol=Fine-tuned, Ensemble=true, Visual Encoder Pre-training=true2024.07 | 68.1 | 90.2 | 95.2 | 474.7 | — | |
| MagicLens-LBackbone=CoCa-L, Zero-shot=true2024.03 | 67.7 | 87.6 | 92.7 | — | — | |
| CLIPStructure=ViT-L/142026.03 | 67.1 | 89.4 | 94.7 | — | — | |
| COTSPT Pairs=5.3M, Architecture Type=Dual-Encoder, Evaluation Protocol=Fine-tuned2024.07 | 66.9 | 88.8 | 94 | 463.9 | — | |
| DEPT Pairs=5.1M, Architecture Type=Dual-Encoder, Evaluation Protocol=Fine-tuned2024.07 | 66.7 | 88.9 | 94.4 | 460.3 | — | |
| UNITERPre-train # Images=4M, fine-tuned=true2023.12 | 65.7 | 88.6 | 93.8 | — | — | |
| CoCa-L+Backbone=CoCa-L, Zero-shot=true, Source=original paper2024.03 | 65.4 | 85.6 | 91.4 | — | — | |
| CoCa-L*Backbone=CoCa-L, Zero-shot=true, Source=reproduced2024.03 | 65.1 | 85.5 | 91.3 | — | — | |
| MagicLens-BBackbone=CoCa-B, Zero-shot=true2024.03 | 64.8 | 85.5 | 91.2 | — | — | |
| CoCa-B*Backbone=CoCa-B, Zero-shot=true, Source=reproduced2024.03 | 63.8 | 84.7 | 91.2 | — | — | |
| USERVisual Backbone=Faster R-CNN, Text Encoder=BERT, Ensemble=false2026.03 | 63.7 | 87.4 | 93.5 | — | — | |
| Pixel-BERT-X152PT Pairs=5.6M, Architecture Type=Cross-Encoder, Evaluation Protocol=Fine-tuned2024.07 | 63.6 | 87.5 | 93.6 | 458.6 | — | |
| CoCa-B+Backbone=CoCa-B, Zero-shot=true, Source=original paper2024.03 | 63.4 | 84.7 | 90.7 | — | — | |
| MaxMatchVisual Backbone=Faster R-CNN, Text Encoder=BERT, Ensemble=false2026.03 | 63.3 | 87.9 | 93.2 | — | — | |
| HREMVisual Backbone=Faster R-CNN, Text Encoder=BERT, Ensemble=false2026.03 | 62.3 | 87.6 | 93.4 | — | — | |
| COOKIEPT Pairs=5.9M, Architecture Type=Dual-Encoder, Evaluation Protocol=Fine-tuned2024.07 | 61.7 | 86.7 | 92.3 | 446.6 | — | |
| ViLTPT Pairs=9.9M, Architecture Type=Cross-Encoder, Evaluation Protocol=Fine-tuned2024.07 | 61.5 | 86.3 | 92.7 | 439.2 | — | |
| CMSFStructure=frcnn+bert2026.03 | 61.5 | 86.7 | 92.8 | — | — | |
| CMSFVisual Backbone=Faster R-CNN, Text Encoder=BERT, Ensemble=false2026.03 | 61.5 | 86.7 | 92.8 | — | — | |
| LightningDOTPT Pairs=9.5M, Architecture Type=Dual-Encoder, Evaluation Protocol=Fine-tuned2024.07 | 60.1 | 85.1 | 91.8 | 441.2 | — | |
| CHANVisual Backbone=Faster R-CNN, Text Encoder=BERT, Ensemble=false2026.03 | 59.8 | 87.2 | 93.3 | — | — | |
| TERANVisual Backbone=Faster R-CNN, Text Encoder=BERT, Ensemble=true2026.03 | 59.3 | 85.8 | 92.4 | — | — | |
| NAAFVisual Backbone=Fast R-CNN, Text Encoder=BiGRU, Ensemble=false2026.03 | 58.9 | 85.2 | 92 | — | — | |
| CMSFStructure=frcnn+bigru2026.03 | 58.5 | 85.3 | 92.5 | — | — | |
| CMSFVisual Backbone=Fast R-CNN, Text Encoder=BiGRU, Ensemble=false2026.03 | 58.5 | 85.3 | 92.5 | — | — | |
| CLIP-L+Backbone=CLIP-L, Zero-shot=true, Source=original paper2024.03 | 58.4 | 81.5 | 88.1 | — | — | |
| VSE∞Visual Backbone=Faster R-CNN, Text Encoder=BERT, Ensemble=false2026.03 | 58.3 | 85.3 | 92.3 | — | — | |
| VSE∞Visual Backbone=Fast R-CNN, Text Encoder=BiGRU, Ensemble=false2026.03 | 56.6 | 83.6 | 91.4 | — | — | |
| CLIPStructure=ViT-B/322026.03 | 56.3 | 81.7 | 89.4 | — | — | |
| CLIP-L*Backbone=CLIP-L, Zero-shot=true, Source=reproduced2024.03 | 56.2 | 79.3 | 87.3 | — | — | |
| MagicLens-LBackbone=CLIP-L, Zero-shot=true2024.03 | 55.9 | 78.7 | 86.3 | — | — | |
| VSRN++Visual Backbone=Faster R-CNN, Text Encoder=BERT, Ensemble=true2026.03 | 54.7 | 82.9 | 90.9 | — | — | |
| MMCAVisual Backbone=Faster R-CNN, Text Encoder=BERT, Ensemble=false2026.03 | 54 | 82.5 | 90.7 | — | — | |
| NeighborRetr2025.03 | 53.2 | 79.7 | 88.2 | — | 476.4 | |
| VSRNVisual Backbone=Fast R-CNN, Text Encoder=BiGRU, Ensemble=true2026.03 | 53 | 81.1 | 89.4 | — | — | |
| ViLEM2025.03 | 52.6 | 79.4 | 87.2 | — | 474 | |
| CUSA2025.03 | 52.4 | 79.8 | 88.1 | — | 473.2 | |
| CLIP-B*Backbone=CLIP-B, Zero-shot=true, Source=reproduced2024.03 | 51.9 | 76.3 | 83.9 | — | — | |
| PCME2025.03 | 51.2 | 79.1 | 87.5 | — | 474.2 | |
| SOHO2025.03 | 50.6 | 78 | 86.7 | — | 463.7 | |
| SCANVisual Backbone=Fast R-CNN, Text Encoder=BiGRU, Ensemble=true2026.03 | 50.4 | 82.2 | 90 | — | — | |
| MagicLens-BBackbone=CLIP-B, Zero-shot=true2024.03 | 49.5 | 74.5 | 82.5 | — | — | |
| UNITER2025.03 | 48.4 | 76.7 | 85.9 | — | 454.4 | |
| ViSTA2025.03 | 47.8 | 75.8 | 84.5 | — | 453.4 | |
| UniConBackbone=CLIP ViT-B/32, Train time=11.15 s2026.04 | 32.9 | — | 68.5 | — | — | |
| SGD–CLIPBackbone=CLIP ViT-B/32, Train time=1066.60 s2026.04 | 12.8 | — | 41.5 | — | — | |
| UniConBackbone=RN-50 + SBERT, Train time=11.11 s2026.04 | 10.5 | — | 38.8 | — | — | |
| SGD–CLIPBackbone=RN-50 + SBERT, Train time=5121.72 s2026.04 | 5.3 | — | 25.3 | — | — |