Image-text Retrieval on COCO (test)
97.2Recall@1BLIP_FUSECAP
Evaluation results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| BLIP_FUSECAPTraining dataset=enriched2023.05 | 97.2 | — | — | — | — | — | — | 99.5 | 99.9 | — | — | — | — | — | |
| BLIP_FUSECAPTraining dataset=enriched2023.05 | 93 | — | — | — | — | — | — | 97.4 | 98.3 | — | — | — | — | — | |
| BLIP2Training dataset=original2023.05 | 85.4 | — | — | — | — | — | — | 97 | 98.5 | — | — | — | — | — | |
| BLIP-LTraining dataset=original2023.05 | 82.4 | — | — | — | — | — | — | 95.4 | 97.9 | — | — | — | — | — | |
| BLIP†Training dataset=original2023.05 | 75.1 | — | — | — | — | — | — | 92.7 | 96.4 | — | — | — | — | — | |
| EVA-02-CLIP-E/14+zero-shot=true, backbone=ViT-E/14+2023.03 | 68.8 | — | — | — | — | — | — | 87.8 | 92.8 | — | — | — | — | — | |
| BLIP2Training dataset=original2023.05 | 68.3 | — | — | — | — | — | — | 87.7 | 92.6 | — | — | — | — | — | |
| EVA-01-CLIP-g/14+zero-shot=true, backbone=ViT-g/14+2023.03 | 68.2 | — | — | — | — | — | — | 87.5 | 92.5 | — | — | — | — | — | |
| EVA-02-CLIP-E/14zero-shot=true, backbone=ViT-E/142023.03 | 68.1 | — | — | — | — | — | — | 87.7 | 92.8 | — | — | — | — | — | |
| Open CLIP-G/14zero-shot=true, backbone=ViT-G/142023.03 | 67.3 | — | — | — | — | — | — | 86.9 | 92.6 | — | — | — | — | — | |
| Open CLIP-g/14zero-shot=true, backbone=ViT-g/142023.03 | 66.4 | — | — | — | — | — | — | 86 | 91.8 | — | — | — | — | — | |
| Open CLIP-H/14zero-shot=true, backbone=ViT-H/142023.03 | 66 | — | — | — | — | — | — | 86.1 | 91.9 | — | — | — | — | — | |
| BLIP-LTraining dataset=original2023.05 | 65.2 | — | — | — | — | — | — | 86.3 | 91.8 | — | — | — | — | — | |
| EVA-02-CLIP-L/14+zero-shot=true, backbone=ViT-L/14+2023.03 | 64.1 | — | — | — | — | — | — | 85.2 | 90.8 | — | — | — | — | — | |
| EVA-02-CLIP-L/14zero-shot=true, backbone=ViT-L/142023.03 | 63.7 | — | — | — | — | — | — | 84.3 | 90.4 | — | — | — | — | — | |
| Open CLIP-L/14zero-shot=true, backbone=ViT-L/142023.03 | 62.1 | — | — | — | — | — | — | 83.4 | 90.3 | — | — | — | — | — | |
| EVA-01-CLIP-g/14zero-shot=true, backbone=ViT-g/142023.03 | 61.8 | — | — | — | — | — | — | 83.3 | 90 | — | — | — | — | — | |
| Open CLIP-B/16zero-shot=true, backbone=ViT-B/162023.03 | 59.4 | — | — | — | — | — | — | 81.8 | 88.6 | — | — | — | — | — | |
| EVA-02-CLIP-B/16zero-shot=true, backbone=ViT-B/162023.03 | 58.7 | — | — | — | — | — | — | 80.7 | 88.2 | — | — | — | — | — | |
| BLIP†Training dataset=original2023.05 | 58.2 | — | — | — | — | — | — | 82.4 | 89.2 | — | — | — | — | — | |
| OpenAI CLIP-L/14+zero-shot=true, backbone=ViT-L/14+2023.03 | 57.9 | — | — | — | — | — | — | 81.2 | 87.9 | — | — | — | — | — | |
| OpenAI CLIP-L/14zero-shot=true, backbone=ViT-L/142023.03 | 56.3 | — | — | — | — | — | — | 79.3 | 86.7 | — | — | — | — | — | |
| OpenAI CLIP-B/16zero-shot=true, backbone=ViT-B/162023.03 | 52.4 | — | — | — | — | — | — | 76.8 | 84.7 | — | — | — | — | — | |
| NegCLIPTraining Dataset=COCO, Frozen=false2025.02 | 41 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LABCLIPTraining Dataset=COCO, Frozen=true2025.02 | 41 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CleanFine-tuning=CleanCLIP2026.02 | 40.98 | — | — | — | — | — | — | — | 78.25 | — | — | — | — | — | |
| BadNetsFine-tuning=CleanCLIP2026.02 | 39.78 | — | — | — | — | — | — | — | 77.48 | — | 0.4 | 1.56 | — | — | |
| BadCLIPFine-tuning=CleanCLIP2026.02 | 39.71 | — | — | — | — | — | — | — | 77.79 | — | 58.9 | 95.94 | — | — | |
| mmPoisonFine-tuning=CleanCLIP2026.02 | 39.64 | — | — | — | — | — | — | — | 77.21 | — | 0.01 | 0.06 | — | — | |
| SSBAFine-tuning=CleanCLIP2026.02 | 39.61 | — | — | — | — | — | — | — | 77.29 | — | 0.01 | 0.29 | — | — | |
| BlendedFine-tuning=CleanCLIP2026.02 | 39.58 | — | — | — | — | — | — | — | 77.31 | — | 0.06 | 0.42 | — | — | |
| VLTrojanFine-tuning=CleanCLIP2026.02 | 39.57 | — | — | — | — | — | — | — | 77.94 | — | 0.01 | 0.04 | — | — | |
| BadEncoderFine-tuning=CleanCLIP2026.02 | 39.56 | — | — | — | — | — | — | — | 77.32 | — | 0.31 | 0.73 | — | — | |
| INACTIVEFine-tuning=CleanCLIP2026.02 | 39.46 | — | — | — | — | — | — | — | 77.54 | — | 0.46 | 1.02 | — | — | |
| TrojVQAFine-tuning=CleanCLIP2026.02 | 39.34 | — | — | — | — | — | — | — | 77.65 | — | 1.96 | 2.88 | — | — | |
| WaNetFine-tuning=CleanCLIP2026.02 | 39.24 | — | — | — | — | — | — | — | 77.12 | — | 0.04 | 0.58 | — | — | |
| MABAFine-tuning=CleanCLIP2026.02 | 39.24 | — | — | — | — | — | — | — | 77.45 | — | 0.68 | 4.02 | — | — | |
| SIGFine-tuning=CleanCLIP2026.02 | 39.12 | — | — | — | — | — | — | — | 77.25 | — | 1 | 2.59 | — | — | |
| BadCLIP++Fine-tuning=CleanCLIP2026.02 | 39.01 | — | — | — | — | — | — | — | 77.63 | — | 96.62 | 99.82 | — | — | |
| LABCLIPTraining Dataset=CC3M, Frozen=true2025.02 | 34 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CLIPTraining Dataset=-, Frozen=-2025.02 | 30 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ALBEF2022.06 | — | 56.8 | 81.5 | 89.2 | 73.1 | 91.4 | 96 | — | — | — | — | — | — | — | |
| ALBEF-BPretrain Images=4M, Retrieval Strategy=re-ranking2022.06 | — | 56.8 | — | — | 73.1 | — | — | — | — | — | — | — | — | — | |
| ALBEF(14M)Pre-training Data Scale=> 10M images, Evaluation Protocol=fine-tuned2022.11 | — | 60.7 | 84.3 | 90.5 | — | — | — | — | — | — | — | — | — | — | |
| ALBEF(14M)Pre-training Data Scale=> 10M images, Evaluation Protocol=fine-tuned2022.11 | — | — | — | — | 77.6 | 94.3 | 97.2 | — | — | — | — | — | — | — | |
| ALBEF(4M)Pre-training Data Scale=< 10M images, Evaluation Protocol=fine-tuned2022.11 | — | 56.8 | 81.5 | 89.2 | — | — | — | — | — | — | — | — | — | — | |
| ALBEF(4M)Pre-training Data Scale=< 10M images, Evaluation Protocol=fine-tuned2022.11 | — | — | — | — | 73.1 | 91.4 | 96 | — | — | — | — | — | — | — | |
| ALBEF(4M)Pre-train images=4M2023.02 | — | — | — | — | — | — | — | 91.4 | — | — | — | — | — | — | |
| ALIGNPre-training Data Scale=> 10M images, Evaluation Protocol=fine-tuned2022.11 | — | 59.9 | 83.3 | 89.8 | — | — | — | — | — | — | — | — | — | — | |
| ALIGNPre-training Data Scale=> 10M images, Evaluation Protocol=fine-tuned2022.11 | — | — | — | — | 77 | 93.5 | 96.9 | — | — | — | — | — | — | — | |
| BLIP_CapFilt-LPretrain Images=129M, Retrieval Strategy=re-ranking2022.06 | — | 64.1 | — | — | 81.2 | — | — | — | — | — | — | — | — | — | |
| CovMatch# Pairs=5002026.06 | — | — | — | — | — | — | — | — | — | 11.2 | — | — | 9.9 | 12.6 | |
| CovMatch# Pairs=10002026.06 | — | — | — | — | — | — | — | — | — | 14.2 | — | — | 9.6 | 22.1 | |
| CovMatch# Pairs=100, Compression rate=0.8‰2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | 7.4 | |
| CovMatch# Pairs=200, Compression rate=1.7‰2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | 9.2 | |
| CovMatch# Pairs=500, Compression rate=4.4‰2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | 8.3 | |
| DBP#Train=5.3B2024.06 | — | — | — | — | — | — | — | — | — | 48.4 | — | — | — | — | |
| DBP#Train=3.6B2024.06 | — | — | — | — | — | — | — | — | — | 45.7 | — | — | — | — | |
| EDGE# Pairs=5002026.06 | — | — | — | — | — | — | — | — | — | 7.9 | — | — | 6.5 | 9.4 | |
| EDGE# Pairs=10002026.06 | — | — | — | — | — | — | — | — | — | 11.1 | — | — | 9.6 | 12.6 | |
| FIBER-BPretrain Images=4M, Retrieval Strategy=dual encoder2022.06 | — | 58.01 | — | — | 75.38 | — | — | — | — | — | — | — | — | — | |
| FIBER-ITCstrategy=dual-encoder2022.06 | — | 58.01 | 83.45 | 90.11 | 75.38 | 94.04 | 97.36 | — | — | — | — | — | — | — | |
| FIBER-ITC+ITM Ensemblestrategy=ensemble2022.06 | — | 69.73 | 90.66 | 94.59 | 80.1 | 95.6 | 97.98 | — | — | — | — | — | — | — | |
| FIBER-ITMstrategy=fusion-encoder2022.06 | — | 59.03 | 84.04 | 91.03 | 75.14 | 93.88 | 97.36 | — | — | — | — | — | — | — | |
| FIBER-Rerank-10strategy=re-ranking, top-k=102022.06 | — | 68.71 | 87.69 | 90.09 | 79.66 | 95.34 | 97.36 | — | — | — | — | — | — | — | |
| FIBER-Rerank-100strategy=re-ranking, top-k=1002022.06 | — | 69.63 | 90.54 | 94.47 | 80.06 | 95.6 | 97.96 | — | — | — | — | — | — | — | |
| FIBER-Rerank-20strategy=re-ranking, top-k=202022.06 | — | 69.32 | 89.52 | 93.33 | 79.78 | 95.2 | 97.66 | — | — | — | — | — | — | — | |
| FIBER-Rerank-50strategy=re-ranking, top-k=502022.06 | — | 69.58 | 90.41 | 94.35 | 79.98 | 95.4 | 97.76 | — | — | — | — | — | — | — | |
| Forgetting# Pairs=100, Compression rate=0.8‰2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | 2.7 | |
| Forgetting# Pairs=200, Compression rate=1.7‰2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.9 | |
| Forgetting# Pairs=500, Compression rate=4.4‰2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | 7.8 | |
| Herding# Pairs=100, Compression rate=0.8‰2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | 4 | |
| Herding# Pairs=200, Compression rate=1.7‰2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | 6.5 | |
| Herding# Pairs=500, Compression rate=4.4‰2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | 11 | |
| JEST++#Train=1.3B2024.06 | — | — | — | — | — | — | — | — | — | 54.8 | — | — | — | — | |
| K-Center# Pairs=100, Compression rate=0.8‰2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | 4.2 | |
| K-Center# Pairs=200, Compression rate=1.7‰2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | 6.7 | |
| K-Center# Pairs=500, Compression rate=4.4‰2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | 12 | |
| LAION-440M#Train=12.8B2024.06 | — | — | — | — | — | — | — | — | — | 48.1 | — | — | — | — | |
| LoRS# Pairs=100, Compression rate=0.8‰2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | 11.7 | |
| LoRS# Pairs=200, Compression rate=1.7‰2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | 13.7 | |
| LoRS# Pairs=500, Compression rate=4.4‰2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | 17.2 | |
| METERPre-training Data Scale=< 10M images, Evaluation Protocol=fine-tuned2022.11 | — | 57.08 | 82.66 | 90.07 | — | — | — | — | — | — | — | — | — | — | |
| METERPre-training Data Scale=< 10M images, Evaluation Protocol=fine-tuned2022.11 | — | — | — | — | 76.16 | 93.16 | 96.82 | — | — | — | — | — | — | — | |
| METER-Swin-BPretrain Images=4M, Retrieval Strategy=fusion encoder2022.06 | — | 54.85 | — | — | 72.96 | — | — | — | — | — | — | — | — | — | |
| MTT-VL# Pairs=100, Compression rate=0.8‰2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | 9.4 | |
| MTT-VL# Pairs=200, Compression rate=1.7‰2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | 11.5 | |
| MTT-VL# Pairs=500, Compression rate=4.4‰2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | 16.1 | |
| PixelBERTPre-training Data Scale=< 10M images, Evaluation Protocol=fine-tuned2022.11 | — | 50.1 | 77.6 | 86.2 | — | — | — | — | — | — | — | — | — | — | |
| PixelBERTPre-training Data Scale=< 10M images, Evaluation Protocol=fine-tuned2022.11 | — | — | — | — | 63.6 | 87.5 | 93.6 | — | — | — | — | — | — | — | |
| PixelBERT2023.02 | — | — | — | — | — | — | — | 87.5 | — | — | — | — | — | — | |
| RAHA# Pairs=5002026.06 | — | — | — | — | — | — | — | — | — | 13.7 | — | — | 12.6 | 14.9 | |
| RAHA# Pairs=10002026.06 | — | — | — | — | — | — | — | — | — | 18.6 | — | — | 16.4 | 20.8 | |
| RAHA# Pairs=100, Compression rate=0.8‰2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | 7.7 | |
| RAHA# Pairs=200, Compression rate=1.7‰2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | 11 | |
| RAHA# Pairs=500, Compression rate=4.4‰2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | 14.9 | |
| Random# Pairs=100, Compression rate=0.8‰2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | 4 | |
| Random# Pairs=200, Compression rate=1.7‰2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | 6.4 | |
| Random# Pairs=500, Compression rate=4.4‰2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | 11.5 | |
| SCLPre-training Data Scale=< 10M images, Evaluation Protocol=fine-tuned2022.11 | — | 60.14 | 84.56 | 91.45 | — | — | — | — | — | — | — | — | — | — |