Text-to-Image Retrieval on Flickr30K 1K (test)
93.3R@1ERNIE-ViL 2.0
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| ERNIE-ViL 2.0Fine-tuned=true, # Pre-train images=29M2022.09 | 93.3 | 99.4 | 99.8 | — | — | — | |
| NeighborRetr2025.03 | 92.7 | 99.3 | 99.8 | — | 564.8 | — | |
| ViLEM2025.03 | 92.4 | 99.2 | 99.7 | — | 561 | — | |
| X2-VLM_large# Params=593M2022.11 | 91.8 | — | — | — | — | — | |
| X2-VLM-large# Params=593M, Pre-training Data=More Data, Retrieval Strategy=Fusion module2022.11 | 91.8 | 98.6 | 99.5 | — | — | — | |
| SEPSVisual Encoder=Swin-Base-384, Textual Encoder=BERT-base, Image Resolution=384x384, Patch Size=12x12, Fine-Grained Alignment (FG)=true2025.11 | 91.6 | 99.4 | 99.8 | — | 581.9 | — | |
| I-CLIP2025.03 | 91.2 | 99.2 | 99.7 | — | 552.2 | — | |
| X2-VLM-large# Params=593M, Pre-training Data=4M, Retrieval Strategy=Fusion module2022.11 | 91.1 | 98.6 | 99.4 | — | — | — | |
| CUSA2025.03 | 90.8 | 99.1 | 99.7 | — | 560.2 | — | |
| X2-VLM-base# Params=255M, Pre-training Data=More Data, Retrieval Strategy=Fusion module2022.11 | 90.4 | 98.2 | 99.3 | — | — | — | |
| BEIT-3Architecture Type=Dual-encoder, Evaluation Protocol=Finetuning2022.08 | 90.3 | 98.7 | 99.5 | — | — | — | |
| BEiT-3Pre-training Images=35M+, Parameters=1.9B, Evaluation Protocol=Fine-tuned2022.12 | 90.3 | 98.7 | 99.5 | 97.7 | — | — | |
| BEIT-3# Params=1.9B2022.11 | 90.3 | — | — | — | — | — | |
| BEiT-3# Params=1.9B2022.11 | 90.3 | — | — | — | — | — | |
| X2-VLM-base# Params=255M, Pre-training Data=4M, Retrieval Strategy=Fusion module2022.11 | 90 | 98.6 | 99.3 | — | — | — | |
| BLIP-2 ViT-gModel Category=Dual encoder + Fusion encoder reranking, #Trainable Params=1.2B, Evaluation Protocol=Zero-shot2023.01 | 89.7 | 98.1 | 98.9 | — | — | — | |
| BLIP-2multi-lingual=false, Zero-shot=true2023.12 | 89.7 | 98.1 | 98.9 | — | — | — | |
| ONE-PEACEEvaluation Setting=Fine-tuning2023.05 | 89.6 | 98 | 99.1 | — | — | — | |
| ViSTA2025.03 | 89.5 | 98.4 | 99.6 | — | 554.4 | — | |
| SEPSVisual Encoder=ViT-Base-384, Textual Encoder=BERT-base, Image Resolution=384x384, Patch Size=24x24, Fine-Grained Alignment (FG)=true2025.11 | 89.3 | 99.3 | 99.5 | — | 571.5 | — | |
| BEIT-3Evaluation Setting=Fine-tuning2023.05 | 89.1 | 98.6 | 99.3 | — | — | — | |
| BLIP-2 ViT-LModel Category=Dual encoder + Fusion encoder reranking, #Trainable Params=474M, Evaluation Protocol=Zero-shot2023.01 | 88.6 | 97.6 | 98.9 | — | — | — | |
| CSIC2025.03 | 88.5 | 98.1 | 99.4 | — | 551.6 | — | |
| mPLUGPretrain data=14M2022.05 | 88.4 | 97.9 | 99.1 | — | — | — | |
| IAIS2025.03 | 88.3 | 98.4 | 99.4 | — | 552 | — | |
| SEPSVisual Encoder=Swin-Base-224, Textual Encoder=BERT-base, Image Resolution=224x224, Patch Size=7x7, Fine-Grained Alignment (FG)=true2025.11 | 88 | 98.9 | 99.6 | — | 572 | — | |
| FlorenceEvaluation Protocol=Fine-tuned2021.11 | 87.9 | 98.1 | — | — | — | — | |
| FlorenceArchitecture Type=Dual-encoder, Evaluation Protocol=Finetuning2022.08 | 87.9 | 98.1 | — | — | — | — | |
| FlorenceFine-tuned=true, # Pre-train images=900M2022.09 | 87.9 | 98.1 | — | — | — | — | |
| FlorenceEvaluation Setting=Fine-tuning2023.05 | 87.9 | 98.1 | — | — | — | — | |
| OmniVLEvaluation Setting=Fine-tuning2023.05 | 87.9 | 97.8 | 99.1 | — | — | — | |
| FlorencePre-training Images=900M, Parameters=893M, Evaluation Protocol=Fine-tuned2022.12 | 87.9 | 98.1 | — | — | — | — | |
| FlorencePretrain data=0.9B2022.05 | 87.9 | 98.1 | — | — | — | — | |
| OmniVL# Params=288M, Pre-training Data=More Data, Retrieval Strategy=Fusion module2022.11 | 87.9 | 97.8 | 99.1 | — | — | — | |
| PTP-BLIPPre-training Images=14M, Parameters=220M, Evaluation Protocol=Fine-tuned2022.12 | 87.7 | 98.2 | 99.3 | 97 | — | — | |
| BLIP_ViT-LPre-train # Images=129M2022.01 | 87.6 | 97.7 | 99 | — | — | — | |
| BLIPArchitecture Type=Dual encoder + Fusion encoder reranking, Evaluation Protocol=Finetuning2022.08 | 87.6 | 97.7 | 99 | — | — | — | |
| BLIPPretrain data=129M2022.05 | 87.6 | 97.7 | 99 | — | — | — | |
| BLIP-large# Params=452M, Pre-training Data=More Data, Retrieval Strategy=Fusion module2022.11 | 87.6 | 97.7 | 99 | — | — | — | |
| BLIP_CapFilt-LPre-train # Images=129M2022.01 | 87.5 | 97.7 | 98.9 | — | — | — | |
| BLIPPre-train # Images=129M2022.01 | 87.3 | 97.6 | 98.9 | — | — | — | |
| BLIP-base# Params=240M, Pre-training Data=More Data, Retrieval Strategy=Fusion module2022.11 | 87.3 | 97.6 | 98.9 | — | — | — | |
| BLIPPre-train # Images=14M2022.01 | 87.2 | 97.5 | 98.8 | — | — | — | |
| BLIPPre-training Images=14M, Parameters=220M, Evaluation Protocol=Fine-tuned2022.12 | 87.2 | 97.5 | 98.8 | 96.7 | — | — | |
| BLIPPretrain data=14M2022.05 | 87.2 | 97.5 | 98.8 | — | — | — | |
| BLIPPre-train # Images=14M, fine-tuned=true2023.12 | 87.2 | 97.5 | 98.8 | — | — | — | |
| FLIPEvaluation Protocol=Fine-tuned2021.11 | 87.1 | 97.7 | — | — | — | — | |
| FILIPArchitecture Type=Dual-encoder, Evaluation Protocol=Finetuning2022.08 | 87.1 | 97.7 | 99.1 | — | — | — | |
| FILIPFine-tuned=true, # Pre-train images=340M2022.09 | 87.1 | 97.7 | 99.1 | — | — | — | |
| FILIPEvaluation Setting=Fine-tuning2023.05 | 87.1 | 97.7 | 99.1 | — | — | — | |
| FILIPPre-training Images=340M, Parameters=787M, Evaluation Protocol=Fine-tuned2022.12 | 87.1 | 97.7 | 99.1 | 96.8 | — | — | |
| FLIP# Params=420M, Pre-training Data=More Data, Retrieval Strategy=Dual-encoder2022.11 | 87.1 | 97.7 | 99.1 | — | — | — | |
| UncompressedReduce Ratio=None, GFLOPS=153.2, Base Model=BLIP2024.03 | 86.9 | 97.3 | 98.7 | — | — | — | |
| UncompressedModel=BLIP, Pruning Mode=/, Pruning Ratio=/, GFLOPs=91.652026.04 | 86.9 | 97.3 | — | — | — | — | |
| UncompressedModel=BLIP, Pruning Mode=None, Pruning Ratio=None, GFLOPs=91.652026.04 | 86.9 | 97.3 | — | — | — | — | |
| SEPSVisual Encoder=ViT-Base-224, Textual Encoder=BERT-base, Image Resolution=224x224, Patch Size=14x14, Fine-Grained Alignment (FG)=true2025.11 | 86.9 | 98.1 | 99.2 | — | 560.9 | — | |
| BLIPModel Category=Dual encoder + Fusion encoder reranking, #Trainable Params=446M, Evaluation Protocol=Zero-shot2023.01 | 86.7 | 97.3 | 98.7 | — | — | — | |
| UncompressedBackbone=CLIP, Reduce Ratio=/, GFLOPS=395.72024.03 | 86.6 | 97.8 | 99.1 | — | — | — | |
| UncompressedModel=CLIP, Pruning Mode=/, Pruning Ratio=/, GFLOPs=197.82026.04 | 86.6 | 97.8 | — | — | — | — | |
| UncompressedModel=CLIP, Pruning Mode=None, Pruning Ratio=None, GFLOPs=197.82026.04 | 86.6 | 97.8 | — | — | — | — | |
| X-VLM oursPre-train=true, Data size=4M2022.10 | 86.3 | 97.4 | 99 | — | — | — | |
| X-VLMPre-train=true, Data size=4M2022.10 | 86.1 | 97.4 | 98.7 | — | — | — | |
| ALBEFPre-train # Images=14M2022.01 | 85.6 | 97.5 | 98.9 | — | — | — | |
| ALBEFArchitecture Type=Dual encoder + Fusion encoder reranking, Evaluation Protocol=Finetuning2022.08 | 85.6 | 97.5 | 98.9 | — | — | — | |
| ALBEFFine-tuned=true, # Pre-train images=14M2022.09 | 85.6 | 97.5 | 98.9 | — | — | — | |
| ALBEFPre-train=true, Data size=14M2022.10 | 85.6 | 97.5 | 98.9 | — | — | — | |
| ALBEFPre-training Images=14M, Parameters=210M, Evaluation Protocol=Fine-tuned2022.12 | 85.6 | 97.5 | 98.9 | 96.3 | — | — | |
| ALBEFPretrain data=14M2022.05 | 85.6 | 97.5 | 98.9 | — | — | — | |
| ALBEFPre-train # Images=14M, fine-tuned=true2023.12 | 85.6 | 97.5 | 98.9 | — | — | — | |
| CEPT Pairs=15.2M, Architecture Type=Cross-Encoder, Evaluation Protocol=Fine-tuned2024.07 | 85.6 | 97.5 | 98.9 | — | — | 577.7 | |
| ERNIE-ViL 2.0Pre-train images=29M, Protocol=Fine-tuned on MSCOCO dataset2022.09 | 85 | 97 | 98.3 | — | — | — | |
| InternVL-Gmulti-lingual=true, Zero-shot=true2023.12 | 85 | 97 | 98.6 | — | — | — | |
| ALIGNEvaluation Protocol=Fine-tuned2021.02 | 84.9 | 97.4 | 98.6 | — | — | — | |
| ALIGNEvaluation Protocol=Fine-tuned2021.11 | 84.9 | 97.4 | — | — | — | — | |
| ALIGNPre-train # Images=1.8B2022.01 | 84.9 | 97.4 | 98.6 | — | — | — | |
| ALIGNArchitecture Type=Dual-encoder, Evaluation Protocol=Finetuning2022.08 | 84.9 | 97.4 | 98.6 | — | — | — | |
| ALIGNFine-tuned=true, # Pre-train images=1.8B2022.09 | 84.9 | 97.4 | 98.6 | — | — | — | |
| ALIGNEvaluation Setting=Fine-tuning2023.05 | 84.9 | 97.4 | 98.6 | — | — | — | |
| ALIGNPre-training Images=1.8B, Parameters=820M, Evaluation Protocol=Fine-tuned2022.12 | 84.9 | 97.4 | 98.6 | 96 | — | — | |
| ALIGNPretrain data=1.8B2022.05 | 84.9 | 97.4 | 98.6 | — | — | — | |
| ALIGN# Params=490M, Pre-training Data=More Data, Retrieval Strategy=Dual-encoder2022.11 | 84.9 | 97.4 | 98.6 | — | — | — | |
| MAFAPre-train # Images=4M, fine-tuned=true2023.12 | 84.9 | 96.5 | 98 | — | — | — | |
| FILIPPre-train images=340M, Protocol=Fine-tuned on MSCOCO dataset2022.09 | 84.7 | 97 | 98.7 | — | — | — | |
| MAFA (4M-Clean)Pre-train # Images=4M, fine-tuned=true2023.12 | 84.6 | 96.4 | 98.1 | — | — | — | |
| VLMoPretrain data=4M2022.05 | 84.5 | 97.3 | 98.6 | — | — | — | |
| VLMo-large# Params=562M, Pre-training Data=4M, Retrieval Strategy=Dual-encoder2022.11 | 84.5 | 97.3 | 98.6 | — | — | — | |
| PTP-BLIPPre-training Images=4M, Parameters=220M, Evaluation Protocol=Fine-tuned2022.12 | 84.2 | 96.6 | 98.6 | 95.9 | — | — | |
| TCLPre-train # Images=4M, fine-tuned=true2023.12 | 84 | 96.7 | 98.5 | — | — | — | |
| VL-BEIT# Params=175M, Pre-training Data=4M, Retrieval Strategy=Fusion module2022.11 | 83.9 | — | — | — | — | — | |
| OmniVL# Params=288M, Pre-training Data=4M, Retrieval Strategy=Fusion module2022.11 | 83.4 | 97 | 98.6 | — | — | — | |
| MADTPBackbone=CLIP, Reduce Ratio=0.5, GFLOPS=178.82024.03 | 83.3 | 97 | 98.5 | — | — | — | |
| GRIT-VLP*Pre-train # Images=4M, fine-tuned=true2023.12 | 82.9 | 96.2 | 97.9 | — | — | — | |
| ALBEFPre-train images=4M, Protocol=Fine-tuned on MSCOCO dataset2022.09 | 82.8 | 96.3 | 98.1 | — | — | — | |
| ALBEFModel Category=Dual encoder + Fusion encoder reranking, #Trainable Params=233M, Evaluation Protocol=Zero-shot2023.01 | 82.8 | 96.3 | 98.1 | — | — | — | |
| ALBEF# Params=210M, Pre-training Data=4M, Retrieval Strategy=Fusion module2022.11 | 82.8 | 96.7 | 98.4 | — | — | — | |
| ALBEFPre-train # Images=4M, fine-tuned=true2023.12 | 82.8 | 96.7 | 98.4 | — | — | — | |
| ALBEFPT Pairs=5.1M, Architecture Type=Cross-Encoder, Evaluation Protocol=Fine-tuned2024.07 | 82.8 | 96.7 | 98.4 | — | — | 571.4 | |
| BLIP* (4M-Clean)Pre-train # Images=4M, fine-tuned=true2023.12 | 82.6 | 96.2 | 98.3 | — | — | — | |
| BLIPPre-training Images=4M, Parameters=220M, Evaluation Protocol=Fine-tuned, Implementation=Own implementation2022.12 | 82.5 | 96.4 | 98.2 | 95 | — | — | |
| MaMMUTimage model size=630M, evaluation protocol=zero-shot, model type=dual-encoder2023.03 | 82.5 | 96 | 98 | — | — | — |