Text-to-Image Retrieval on MSCOCO 5K (test)
69.5R@1NeighborRetr
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| NeighborRetr2025.03 | 69.5 | 90.5 | 95.3 | — | — | — | — | — | 476.4 | |
| ViLEM2025.03 | 69 | 90.7 | 95.1 | — | — | — | — | — | 474 | |
| CUSA2025.03 | 67.9 | 90.3 | 94.7 | — | — | — | — | — | 473.2 | |
| X2-VLM_large# Params=593M2022.11 | 67.7 | — | — | — | — | — | — | — | — | |
| X2-VLM-large# Params=593M, Pre-training Data=More Data, Retrieval Strategy=Fusion module2022.11 | 67.7 | 87.5 | 92.5 | — | — | — | — | — | — | |
| BEIT-3Architecture Type=Dual-encoder, Evaluation Protocol=Finetuning2022.08 | 67.2 | 87.7 | 92.8 | — | — | — | — | — | — | |
| BEiT-3Pre-training Images=35M+, Parameters=1.9B, Evaluation Protocol=Fine-tuned2022.12 | 67.2 | 87.7 | 92.8 | — | — | — | 87.8 | — | — | |
| BEIT-3# Params=1.9B2022.11 | 67.2 | — | — | — | — | — | — | — | — | |
| BEiT-3# Params=1.9B2022.11 | 67.2 | — | — | — | — | — | — | — | — | |
| SOHO2025.03 | 66.4 | 88.2 | 93.8 | — | — | — | — | — | 463.7 | |
| X2-VLM-base# Params=255M, Pre-training Data=More Data, Retrieval Strategy=Fusion module2022.11 | 66.2 | 87.1 | 92.2 | — | — | — | — | — | — | |
| mPLUGPretrain data=14M2022.05 | 65.8 | 87.3 | 92.6 | — | — | — | — | — | — | |
| PCME2025.03 | 65.3 | 89.2 | 94.5 | — | — | — | — | — | 474.2 | |
| X2-VLM-large# Params=593M, Pre-training Data=4M, Retrieval Strategy=Fusion module2022.11 | 65.2 | 86.4 | 91.9 | — | — | — | — | — | — | |
| BLIP_ViT-LPre-train # Images=129M2022.01 | 65.1 | 86.3 | 91.8 | — | — | — | — | — | — | |
| BLIPArchitecture Type=Dual encoder + Fusion encoder reranking, Evaluation Protocol=Finetuning2022.08 | 65.1 | 86.3 | 91.8 | — | — | — | — | — | — | |
| BLIPPretrain data=129M2022.05 | 65.1 | 86.3 | 91.8 | — | — | — | — | — | — | |
| BLIP-large# Params=452M, Pre-training Data=More Data, Retrieval Strategy=Fusion module2022.11 | 65.1 | 86.3 | 91.8 | — | — | — | — | — | — | |
| PTP-BLIPPre-training Images=14M, Parameters=220M, Evaluation Protocol=Fine-tuned2022.12 | 64.9 | 87.4 | 92.2 | — | — | — | 86.6 | — | — | |
| OmniVL# Params=288M, Pre-training Data=More Data, Retrieval Strategy=Fusion module2022.11 | 64.8 | 86.1 | 91.6 | — | — | — | — | — | — | |
| BLIPPre-train # Images=129M2022.01 | 64.3 | 85.7 | 91.5 | — | — | — | — | — | — | |
| BLIP-base# Params=240M, Pre-training Data=More Data, Retrieval Strategy=Fusion module2022.11 | 64.3 | 85.7 | 91.5 | — | — | — | — | — | — | |
| BLIP_CapFilt-LPre-train # Images=129M2022.01 | 64.1 | 85.8 | 91.6 | — | — | — | — | — | — | |
| ViSTA2025.03 | 63.9 | 87.8 | 93.6 | — | — | — | — | — | 453.4 | |
| UNITER2019.08 | 63.3 | 87 | 93.1 | — | — | — | — | — | — | |
| UNITER2025.03 | 63.3 | 87 | 93.1 | — | — | — | — | — | 454.4 | |
| FlorenceEvaluation Protocol=Fine-tuned2021.11 | 63.2 | 85.7 | — | — | — | — | — | — | — | |
| FlorenceArchitecture Type=Dual-encoder, Evaluation Protocol=Finetuning2022.08 | 63.2 | 85.7 | — | — | — | — | — | — | — | |
| FlorenceFine-tuned=true, # Pre-train images=900M2022.09 | 63.2 | 85.7 | — | — | — | — | — | — | — | |
| FlorencePre-training Images=900M, Parameters=893M, Evaluation Protocol=Fine-tuned2022.12 | 63.2 | 85.7 | — | — | — | — | — | — | — | |
| FlorencePretrain data=0.9B2022.05 | 63.2 | 85.7 | — | — | — | — | — | — | — | |
| BLIPPre-train # Images=14M2022.01 | 63.1 | 85.3 | 91.1 | — | — | — | — | — | — | |
| BLIPPre-training Images=14M, Parameters=220M, Evaluation Protocol=Fine-tuned2022.12 | 63.1 | 85.3 | 91.1 | — | — | — | 85.5 | — | — | |
| BLIPPretrain data=14M2022.05 | 63.1 | 85.3 | 91.1 | — | — | — | — | — | — | |
| BLIPPre-train # Images=14M, fine-tuned=true2023.12 | 63.1 | 85.3 | 91.1 | — | — | — | — | — | — | |
| X2-VLM-base# Params=255M, Pre-training Data=4M, Retrieval Strategy=Fusion module2022.11 | 62.7 | 84.7 | 90.7 | — | — | — | — | — | — | |
| Unicoder-VL2019.08 | 62.3 | 87.1 | 92.8 | — | — | — | — | — | — | |
| MAFA (4M-Clean)Pre-train # Images=4M, fine-tuned=true2023.12 | 61.6 | 84.5 | 90.4 | — | — | — | — | — | — | |
| VL-BEIT# Params=175M, Pre-training Data=4M, Retrieval Strategy=Fusion module2022.11 | 61.5 | — | — | — | — | — | — | — | — | |
| FLIPEvaluation Protocol=Fine-tuned2021.11 | 61.2 | 84.3 | — | — | — | — | — | — | — | |
| FILIPArchitecture Type=Dual-encoder, Evaluation Protocol=Finetuning2022.08 | 61.2 | 84.3 | 90.6 | — | — | — | — | — | — | |
| FILIPFine-tuned=true, # Pre-train images=340M2022.09 | 61.2 | 84.3 | 90.5 | — | — | — | — | — | — | |
| FILIPPre-training Images=340M, Parameters=787M, Evaluation Protocol=Fine-tuned2022.12 | 61.2 | 84.3 | 90.6 | — | — | — | 84.5 | — | — | |
| FLIP# Params=420M, Pre-training Data=More Data, Retrieval Strategy=Dual-encoder2022.11 | 61.2 | 84.3 | 90.6 | — | — | — | — | — | — | |
| MAFAPre-train # Images=4M, fine-tuned=true2023.12 | 61.2 | 84.3 | 90.3 | — | — | — | — | — | — | |
| MAPPre-training dataset size=< 10M images, Model scale=Base, Fine-tuning=true2022.10 | 60.9 | 86.2 | 93.1 | — | — | — | — | — | — | |
| ALBEF (14M)Pre-training dataset size=> 10M images, Fine-tuning=true2022.10 | 60.7 | 84.3 | 90.5 | — | — | — | — | — | — | |
| ALBEFPre-train # Images=14M2022.01 | 60.7 | 84.3 | 90.5 | — | — | — | — | — | — | |
| ALBEFArchitecture Type=Dual encoder + Fusion encoder reranking, Evaluation Protocol=Finetuning2022.08 | 60.7 | 84.3 | 90.5 | — | — | — | — | — | — | |
| ALBEFFine-tuned=true, # Pre-train images=14M2022.09 | 60.7 | 84.3 | 90.5 | — | — | — | — | — | — | |
| ALBEFPre-training Images=14M, Parameters=210M, Evaluation Protocol=Fine-tuned2022.12 | 60.7 | 84.3 | 90.5 | — | — | — | 84.1 | — | — | |
| ALBEFPretrain data=14M2022.05 | 60.7 | 84.3 | 90.5 | — | — | — | — | — | — | |
| ALBEFPre-train # Images=14M, fine-tuned=true2023.12 | 60.7 | 84.3 | 90.5 | — | — | — | — | — | — | |
| CEPT Pairs=15.2M, Architecture Type=Cross-Encoder, Evaluation Protocol=Fine-tuned2024.07 | 60.7 | 84.3 | 90.5 | — | — | — | — | 504.6 | — | |
| VLMoPretrain data=4M2022.05 | 60.6 | 84.4 | 91 | — | — | — | — | — | — | |
| VLMo-large# Params=562M, Pre-training Data=4M, Retrieval Strategy=Dual-encoder2022.11 | 60.6 | 84.4 | 91 | — | — | — | — | — | — | |
| OscarEvaluation Protocol=Fine-tuned2021.02 | 59.9 | 83.3 | 89.8 | — | — | — | — | — | — | |
| ALIGNEvaluation Protocol=Fine-tuned2021.02 | 59.9 | 83.3 | 89.8 | — | — | — | — | — | — | |
| ALIGNEvaluation Protocol=Fine-tuned2021.11 | 59.9 | 83.3 | — | — | — | — | — | — | — | |
| ALIGNPre-train # Images=1.8B2022.01 | 59.9 | 83.3 | 89.8 | — | — | — | — | — | — | |
| ALIGNArchitecture Type=Dual-encoder, Evaluation Protocol=Finetuning2022.08 | 59.9 | 83.3 | 89.8 | — | — | — | — | — | — | |
| ALIGNFine-tuned=true, # Pre-train images=1.8B2022.09 | 59.9 | 83.3 | 89.8 | — | — | — | — | — | — | |
| ALIGNPre-training Images=1.8B, Parameters=820M, Evaluation Protocol=Fine-tuned2022.12 | 59.9 | 83.3 | 89.8 | — | — | — | 83.4 | — | — | |
| ALIGNPretrain data=1.8B2022.05 | 59.9 | 83.3 | 89.8 | — | — | — | — | — | — | |
| ALIGN# Params=490M, Pre-training Data=More Data, Retrieval Strategy=Dual-encoder2022.11 | 59.9 | 83.3 | 89.8 | — | — | — | — | — | — | |
| GRIT-VLP*Pre-train # Images=4M, fine-tuned=true2023.12 | 59.6 | 83.3 | 89.9 | — | — | — | — | — | — | |
| ERNIE-ViL 2.0Fine-tuned=true, # Pre-train images=29M2022.09 | 59.5 | 83.4 | 90.1 | — | — | — | — | — | — | |
| PTP-BLIPPre-training Images=4M, Parameters=220M, Evaluation Protocol=Fine-tuned2022.12 | 59.4 | 83.4 | 90.4 | — | — | — | 83.7 | — | — | |
| TCLPre-training dataset size=< 10M images, Model scale=Base, Fine-tuning=true2022.10 | 59 | 83.2 | 89.9 | — | — | — | — | — | — | |
| TCLPre-train # Images=4M, fine-tuned=true2023.12 | 59 | 83.2 | 89.9 | — | — | — | — | — | — | |
| BLIP* (4M-Clean)Pre-train # Images=4M, fine-tuned=true2023.12 | 58.9 | 83.1 | 89.6 | — | — | — | — | — | — | |
| VinVL-LargePre-training dataset size=< 10M images, Model scale=Large, Fine-tuning=true2022.10 | 58.8 | 83.5 | 90.3 | — | — | — | — | — | — | |
| VinVLArchitecture Type=Fusion-encoder, Evaluation Protocol=Finetuning2022.08 | 58.8 | 83.5 | 90.3 | — | — | — | — | — | — | |
| Teacher Cross-Encoder (CE)Training Stage=Post Pre-train, Evaluation Protocol=Without fine-tuning2024.07 | 58.7 | 81.7 | 88.5 | — | — | — | — | 487.1 | — | |
| OmniVL# Params=288M, Pre-training Data=4M, Retrieval Strategy=Fusion module2022.11 | 58.5 | 82.6 | 89.5 | — | — | — | — | — | — | |
| VinVL-Base# PT Pairs=8.9M, Architecture=Single-Stream, Visual Encoder Pre-training=standard, Ensemble=false2022.04 | 58.1 | 83.2 | 90.1 | — | — | — | — | — | — | |
| VinVLPre-training Images=4M, Parameters=157M, Evaluation Protocol=Fine-tuned2022.12 | 58.1 | 83.2 | 90.1 | — | — | — | 82.5 | — | — | |
| VinVL-basePT Pairs=8.9M, Architecture Type=Cross-Encoder, Evaluation Protocol=Fine-tuned2024.07 | 58.1 | 83.2 | 90.1 | — | — | — | — | 494.9 | — | |
| CEPT Pairs=5.1M, Architecture Type=Cross-Encoder, Evaluation Protocol=Fine-tuned2024.07 | 58 | 82.8 | 89.7 | — | — | — | — | 493.7 | — | |
| OscarEvaluation Protocol=Fine-tuned2021.11 | 57.5 | 82.8 | — | — | — | — | — | — | — | |
| OscarArchitecture Type=Fusion-encoder, Evaluation Protocol=Finetuning2022.08 | 57.5 | 82.8 | 89.8 | — | — | — | — | — | — | |
| OSCARFine-tuned=true, # Pre-train images=4M2022.09 | 57.5 | 82.8 | 89.8 | — | — | — | — | — | — | |
| BLIPPre-training Images=4M, Parameters=220M, Evaluation Protocol=Fine-tuned, Implementation=Own implementation2022.12 | 57.4 | 82.1 | 89.5 | — | — | — | 82.3 | — | — | |
| VLMo-base# Params=175M, Pre-training Data=4M, Retrieval Strategy=Dual-encoder2022.11 | 57.2 | 82.6 | 89.8 | — | — | — | — | — | — | |
| METERPre-training dataset size=< 10M images, Model scale=Base, Fine-tuning=true2022.10 | 57.1 | 82.7 | 90.1 | — | — | — | — | — | — | |
| ALBEF (4M)Pre-training dataset size=< 10M images, Model scale=Base, Fine-tuning=true2022.10 | 56.8 | 81.5 | 89.2 | — | — | — | — | — | — | |
| ALBEF# Params=210M, Pre-training Data=4M, Retrieval Strategy=Fusion module2022.11 | 56.8 | 81.5 | 89.2 | — | — | — | — | — | — | |
| ALBEFPre-train # Images=4M, fine-tuned=true2023.12 | 56.8 | 81.5 | 89.2 | — | — | — | — | — | — | |
| ALBEFPT Pairs=5.1M, Architecture Type=Cross-Encoder, Evaluation Protocol=Fine-tuned2024.07 | 56.8 | 81.5 | 89.2 | — | — | — | — | 488 | — | |
| SCG2019.08 | 56.6 | 84.5 | 92 | — | — | — | — | — | — | |
| Teacher Cross-Encoder (CE)Training Stage=Pre-train from Scratch, Evaluation Protocol=Without fine-tuning2024.07 | 54.8 | 79.7 | 87 | — | — | — | — | 481.9 | — | |
| COOKIE# PT Pairs=5.9M, Architecture=Two-Stream, Visual Encoder Pre-training=940M tagged images, Ensemble=true2022.04 | 54.5 | 81 | 88.2 | — | — | — | — | — | — | |
| COOKIEPT Pairs=5.9M, Architecture Type=Dual-Encoder, Evaluation Protocol=Fine-tuned, Ensemble=true, Visual Encoder Pre-training=true2024.07 | 54.5 | 81 | 88.2 | — | — | — | — | 481.6 | — | |
| OscarQuery dependence=Query-dependent2021.01 | 54 | 80.8 | 88.5 | — | — | — | — | — | — | |
| OSCARPre-train # Images=4M2022.01 | 54 | 80.8 | 88.5 | — | — | — | — | — | — | |
| OSCAR-Base# PT Pairs=6.5M, Architecture=Single-Stream, Visual Encoder Pre-training=standard, Ensemble=false2022.04 | 54 | 80.8 | 88.5 | — | — | — | — | — | — | |
| OSCARPre-training Images=4M, Parameters=155M, Evaluation Protocol=Fine-tuned2022.12 | 54 | 80.8 | 88.5 | — | — | — | — | — | — | |
| OSCARPretrain data=4M2022.05 | 54 | 80.8 | 88.5 | — | — | — | — | — | — | |
| OSCARPre-train # Images=4M, fine-tuned=true2023.12 | 54 | 80.8 | 88.5 | — | — | — | — | — | — | |
| Oscar-basePT Pairs=6.5M, Architecture Type=Cross-Encoder, Evaluation Protocol=Fine-tuned2024.07 | 54 | 80.8 | 88.5 | — | — | — | — | 479.9 | — |