Image-to-Text Retrieval on Flickr30K 1K (test)
99.1R@1X2-VLM-large
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| X2-VLM-large# Params=593M, Pre-training Data=4M, Retrieval Strategy=Fusion module2022.11 | 99.1 | 100 | 100 | — | — | — | |
| X2-VLM_large# Params=593M2022.11 | 98.8 | — | — | — | — | — | |
| X2-VLM-large# Params=593M, Pre-training Data=More Data, Retrieval Strategy=Fusion module2022.11 | 98.8 | 100 | 100 | — | — | — | |
| X2-VLM-base# Params=255M, Pre-training Data=More Data, Retrieval Strategy=Fusion module2022.11 | 98.5 | 100 | 100 | — | — | — | |
| BEIT-3Architecture Type=Dual-encoder, Evaluation Protocol=Finetuning2022.08 | 98 | 100 | 100 | — | — | — | |
| BEiT-3Pre-training Images=35M+, Parameters=1.9B, Evaluation Protocol=Fine-tuned2022.12 | 98 | 100 | 100 | — | — | — | |
| BEIT-3# Params=1.9B2022.11 | 98 | — | — | — | — | — | |
| BEiT-3# Params=1.9B2022.11 | 98 | — | — | — | — | — | |
| ONE-PEACEEvaluation Setting=Fine-tuning2023.05 | 97.6 | 100 | 100 | — | — | — | |
| BLIP-2 ViT-gModel Category=Dual encoder + Fusion encoder reranking, #Trainable Params=1.2B, Evaluation Protocol=Zero-shot2023.01 | 97.6 | 100 | 100 | — | — | — | |
| mPLUGPretrain data=14M2022.05 | 97.6 | 100 | 100 | — | — | — | |
| BEIT-3Evaluation Setting=Fine-tuning2023.05 | 97.5 | 99.9 | 100 | — | — | — | |
| BLIP_ViT-LPre-train # Images=129M2022.01 | 97.4 | 99.8 | 99.9 | — | — | — | |
| BLIPArchitecture Type=Dual encoder + Fusion encoder reranking, Evaluation Protocol=Finetuning2022.08 | 97.4 | 99.8 | 99.9 | — | — | — | |
| BLIPPretrain data=129M2022.05 | 97.4 | 99.8 | 99.9 | — | — | — | |
| X2-VLM-base# Params=255M, Pre-training Data=4M, Retrieval Strategy=Fusion module2022.11 | 97.4 | 99.9 | 100 | — | — | — | |
| BLIP-large# Params=452M, Pre-training Data=More Data, Retrieval Strategy=Fusion module2022.11 | 97.4 | 99.8 | 99.9 | — | — | — | |
| BLIPPre-train # Images=129M2022.01 | 97.3 | 99.9 | 100 | — | — | — | |
| OmniVLEvaluation Setting=Fine-tuning2023.05 | 97.3 | 99.9 | 100 | — | — | — | |
| BLIP-base# Params=240M, Pre-training Data=More Data, Retrieval Strategy=Fusion module2022.11 | 97.3 | 99.9 | 100 | — | — | — | |
| OmniVL# Params=288M, Pre-training Data=More Data, Retrieval Strategy=Fusion module2022.11 | 97.3 | 99.9 | 100 | — | — | — | |
| FlorenceEvaluation Protocol=Fine-tuned2021.11 | 97.2 | 99.9 | — | — | — | — | |
| BLIP_CapFilt-LPre-train # Images=129M2022.01 | 97.2 | 99.9 | 100 | — | — | — | |
| FlorenceArchitecture Type=Dual-encoder, Evaluation Protocol=Finetuning2022.08 | 97.2 | 99.9 | — | — | — | — | |
| FlorenceFine-tuned=true, # Pre-train images=900M2022.09 | 97.2 | 99.9 | — | — | — | — | |
| ERNIE-ViL 2.0Fine-tuned=true, # Pre-train images=29M2022.09 | 97.2 | 100 | 100 | — | — | — | |
| FlorenceEvaluation Setting=Fine-tuning2023.05 | 97.2 | 99.9 | — | — | — | — | |
| FlorencePre-training Images=900M, Parameters=893M, Evaluation Protocol=Fine-tuned2022.12 | 97.2 | 99.9 | — | — | — | — | |
| FlorencePretrain data=0.9B2022.05 | 97.2 | 99.9 | — | — | — | — | |
| X-VLM oursPre-train=true, Data size=4M2022.10 | 97 | 99.6 | 100 | — | — | — | |
| PTP-BLIPPre-training Images=14M, Parameters=220M, Evaluation Protocol=Fine-tuned2022.12 | 97 | 99.9 | 100 | — | — | — | |
| BLIP-2 ViT-LModel Category=Dual encoder + Fusion encoder reranking, #Trainable Params=474M, Evaluation Protocol=Zero-shot2023.01 | 96.9 | 100 | 100 | — | — | — | |
| X-VLMPre-train=true, Data size=4M2022.10 | 96.8 | 99.8 | 100 | — | — | — | |
| UncompressedReduce Ratio=None, GFLOPS=153.2, Base Model=BLIP2024.03 | 96.8 | 99.9 | 100 | — | — | — | |
| UncompressedBackbone=CLIP, Reduce Ratio=/, GFLOPS=395.72024.03 | 96.8 | 100 | 100 | — | — | — | |
| UncompressedModel=BLIP, Pruning Mode=/, Pruning Ratio=/, GFLOPs=91.652026.04 | 96.8 | 99.9 | — | — | — | — | |
| UncompressedModel=CLIP, Pruning Mode=/, Pruning Ratio=/, GFLOPs=197.82026.04 | 96.8 | 100 | — | — | — | — | |
| UncompressedModel=BLIP, Pruning Mode=None, Pruning Ratio=None, GFLOPs=91.652026.04 | 96.8 | 99.9 | — | — | — | — | |
| UncompressedModel=CLIP, Pruning Mode=None, Pruning Ratio=None, GFLOPs=197.82026.04 | 96.8 | 100 | — | — | — | — | |
| BLIPModel Category=Dual encoder + Fusion encoder reranking, #Trainable Params=446M, Evaluation Protocol=Zero-shot2023.01 | 96.7 | 100 | 100 | — | — | — | |
| FLIPEvaluation Protocol=Fine-tuned2021.11 | 96.6 | — | — | — | — | — | |
| BLIPPre-train # Images=14M2022.01 | 96.6 | 99.8 | 100 | — | — | — | |
| FILIPArchitecture Type=Dual-encoder, Evaluation Protocol=Finetuning2022.08 | 96.6 | 100 | 100 | — | — | — | |
| FILIPFine-tuned=true, # Pre-train images=340M2022.09 | 96.6 | 100 | 100 | — | — | — | |
| FILIPEvaluation Setting=Fine-tuning2023.05 | 96.6 | 100 | 100 | — | — | — | |
| BLIPPre-training Images=14M, Parameters=220M, Evaluation Protocol=Fine-tuned2022.12 | 96.6 | 99.8 | 100 | — | — | — | |
| FILIPPre-training Images=340M, Parameters=787M, Evaluation Protocol=Fine-tuned2022.12 | 96.6 | 100 | 100 | — | — | — | |
| BLIPPretrain data=14M2022.05 | 96.6 | 99.8 | 100 | — | — | — | |
| FLIP# Params=420M, Pre-training Data=More Data, Retrieval Strategy=Dual-encoder2022.11 | 96.6 | 100 | 100 | — | — | — | |
| BLIPPre-train # Images=14M, fine-tuned=true2023.12 | 96.6 | 99.8 | 100 | — | — | — | |
| MAFA (4M-Clean)Pre-train # Images=4M, fine-tuned=true2023.12 | 96.2 | 99.9 | 100 | — | — | — | |
| ERNIE-ViL 2.0Pre-train images=29M, Protocol=Fine-tuned on MSCOCO dataset2022.09 | 96.1 | 99.9 | 100 | — | — | — | |
| PTP-BLIPPre-training Images=4M, Parameters=220M, Evaluation Protocol=Fine-tuned2022.12 | 96.1 | 99.8 | 100 | — | — | — | |
| MAFAPre-train # Images=4M, fine-tuned=true2023.12 | 96.1 | 99.8 | 100 | — | — | — | |
| ALBEFPre-train Images=14M, Fine-tuned=true2021.07 | 95.9 | 99.8 | 100 | — | — | — | |
| ALBEFPre-train # Images=14M2022.01 | 95.9 | 99.8 | 100 | — | — | — | |
| ALBEFArchitecture Type=Dual encoder + Fusion encoder reranking, Evaluation Protocol=Finetuning2022.08 | 95.9 | 99.8 | 100 | — | — | — | |
| ALBEFFine-tuned=true, # Pre-train images=14M2022.09 | 95.9 | 99.8 | 100 | — | — | — | |
| ALBEFPre-train=true, Data size=14M2022.10 | 95.9 | 99.8 | 100 | — | — | — | |
| ALBEFPre-training Images=14M, Parameters=210M, Evaluation Protocol=Fine-tuned2022.12 | 95.9 | 99.8 | 100 | — | — | — | |
| ALBEFPretrain data=14M2022.05 | 95.9 | 99.8 | 100 | — | — | — | |
| ALBEFPre-train # Images=14M, fine-tuned=true2023.12 | 95.9 | 99.8 | 100 | — | — | — | |
| CEPT Pairs=15.2M, Architecture Type=Cross-Encoder, Evaluation Protocol=Fine-tuned2024.07 | 95.9 | 99.8 | 100 | — | 577.7 | — | |
| VL-BEIT# Params=175M, Pre-training Data=4M, Retrieval Strategy=Fusion module2022.11 | 95.8 | — | — | — | — | — | |
| GRIT-VLP*Pre-train # Images=4M, fine-tuned=true2023.12 | 95.5 | 99.6 | 99.8 | — | — | — | |
| FILIPPre-train images=340M, Protocol=Fine-tuned on MSCOCO dataset2022.09 | 95.4 | 99.8 | 100 | — | — | — | |
| ALIGNEvaluation Protocol=Fine-tuned2021.02 | 95.3 | 99.8 | 100 | — | — | — | |
| ALIGNPre-train Images=1.2B, Fine-tuned=true2021.07 | 95.3 | 99.8 | 100 | — | — | — | |
| ALIGNEvaluation Protocol=Fine-tuned2021.11 | 95.3 | 99.8 | — | — | — | — | |
| ALIGNPre-train # Images=1.8B2022.01 | 95.3 | 99.8 | 100 | — | — | — | |
| ALIGNArchitecture Type=Dual-encoder, Evaluation Protocol=Finetuning2022.08 | 95.3 | 99.8 | 100 | — | — | — | |
| ALIGNFine-tuned=true, # Pre-train images=1.8B2022.09 | 95.3 | 99.8 | 100 | — | — | — | |
| ALIGNEvaluation Setting=Fine-tuning2023.05 | 95.3 | 99.8 | 100 | — | — | — | |
| ALIGNPre-training Images=1.8B, Parameters=820M, Evaluation Protocol=Fine-tuned2022.12 | 95.3 | 99.8 | 100 | — | — | — | |
| VLMoPretrain data=4M2022.05 | 95.3 | 99.9 | 100 | — | — | — | |
| ALIGNPretrain data=1.8B2022.05 | 95.3 | 99.8 | 100 | — | — | — | |
| VLMo-large# Params=562M, Pre-training Data=4M, Retrieval Strategy=Dual-encoder2022.11 | 95.3 | 99.9 | 100 | — | — | — | |
| ALIGN# Params=490M, Pre-training Data=More Data, Retrieval Strategy=Dual-encoder2022.11 | 95.3 | 99.8 | 100 | — | — | — | |
| MADTPReduce Ratio=0.5, GFLOPS=74.5, Base Model=BLIP2024.03 | 95.1 | 99.5 | 99.7 | — | — | — | |
| BEIT-3Model Category=Dual-encoder models, #Trainable Params=1.9B, Evaluation Protocol=Zero-shot2023.01 | 94.9 | 99.9 | 100 | — | — | — | |
| OmniVL# Params=288M, Pre-training Data=4M, Retrieval Strategy=Fusion module2022.11 | 94.9 | 99.6 | 99.9 | — | — | — | |
| MaMMUTimage model size=630M, evaluation protocol=zero-shot, model type=dual-encoder2023.03 | 94.9 | 99.5 | 99.9 | — | — | — | |
| TCLPre-train # Images=4M, fine-tuned=true2023.12 | 94.9 | 99.5 | 99.8 | — | — | — | |
| VLM2VEC-7BModel category=MLLM-based, Scale=7B2026.04 | 94.6 | 99.5 | 99.8 | — | — | — | |
| CEPT Pairs=5.1M, Architecture Type=Cross-Encoder, Evaluation Protocol=Fine-tuned2024.07 | 94.5 | 99 | 99.7 | — | 563.2 | — | |
| CoMPModel=BLIP, Pruning Mode=C, Pruning Ratio=0.7, GFLOPs=30.472026.04 | 94.4 | 99.6 | — | — | — | — | |
| CoMPModel=BLIP, Pruning Mode=C, Pruning Ratio=0.7, GFLOPs=30.472026.04 | 94.4 | 99.6 | — | — | — | — | |
| ALBEFPre-train Images=4M, Fine-tuned=true2021.07 | 94.3 | 99.4 | 99.8 | — | — | — | |
| ALBEF# Params=210M, Pre-training Data=4M, Retrieval Strategy=Fusion module2022.11 | 94.3 | 99.4 | 99.8 | — | — | — | |
| ALBEFPre-train # Images=4M, fine-tuned=true2023.12 | 94.3 | 99.4 | 99.8 | — | — | — | |
| BLIP* (4M-Clean)Pre-train # Images=4M, fine-tuned=true2023.12 | 94.3 | 99.4 | 99.9 | — | — | — | |
| ALBEFPT Pairs=5.1M, Architecture Type=Cross-Encoder, Evaluation Protocol=Fine-tuned2024.07 | 94.3 | 99.4 | 99.8 | — | 571.4 | — | |
| ALBEF# Pre-train Images=14M, Zero-shot=true2021.07 | 94.1 | 99.5 | 99.7 | — | — | — | |
| ALBEFPre-train images=4M, Protocol=Fine-tuned on MSCOCO dataset2022.09 | 94.1 | 99.5 | 99.7 | — | — | — | |
| ALBEFModel Category=Dual encoder + Fusion encoder reranking, #Trainable Params=233M, Evaluation Protocol=Zero-shot2023.01 | 94.1 | 99.5 | 99.7 | — | — | — | |
| BLIPPre-training Images=4M, Parameters=220M, Evaluation Protocol=Fine-tuned, Implementation=Own implementation2022.12 | 94 | 99.1 | 99.7 | — | — | — | |
| UPopReduce Ratio=0.5, GFLOPS=91.0, Base Model=BLIP2024.03 | 94 | 99.5 | 99.7 | — | — | — | |
| MADTPBackbone=CLIP, Reduce Ratio=0.5, GFLOPS=178.82024.03 | 93.9 | 99.5 | 99.8 | — | — | — | |
| SEPSVisual Encoder=Swin-Base-384, Textual Encoder=BERT-base, Image Resolution=384x384, Patch Size=12x12, Fine-Grained Alignment (FG)=true2025.11 | 93.6 | 98.3 | 99.2 | 581.9 | — | — | |
| UPopBackbone=CLIP, Reduce Ratio=0.5, GFLOPS=201.12024.03 | 93.2 | 99.4 | 99.8 | — | — | — |