Text Retrieval on Flickr30K 1K (test)
98.5R@1X2-VLMbase
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| X2-VLMbaseParameters=255M, Evaluation Protocol=Fine-Tuned, Training Data=More Data2023.01 | 98.5 | 100 | 100 | — | |
| X-FMbaseParameters=327M, Evaluation Protocol=Fine-Tuned, Training Data=More Data2023.01 | 98.1 | 100 | 100 | — | |
| BEIT-3Parameters=1.9B, Evaluation Protocol=Zero-Shot, Training Data=Super-Large2023.01 | 98.1 | 100 | 100 | — | |
| CoCaParameters=2.1B, Evaluation Protocol=Fine-Tuned, Training Data=Super-Large2023.01 | 98 | 100 | 100 | — | |
| BLIP#PT Data=129M2023.02 | 97.4 | 99.8 | 99.9 | — | |
| X2-VLMbaseParameters=255M, Evaluation Protocol=Fine-Tuned, Training Data=Standard2023.01 | 97.4 | 99.9 | 100 | — | |
| X-FMbaseParameters=327M, Evaluation Protocol=Fine-Tuned, Training Data=Standard2023.01 | 97.4 | 100 | 100 | — | |
| BLIP2023.05 | 97.4 | 99.8 | 99.9 | — | |
| BLIPPre-training data size=129M2022.06 | 97.3 | 99.9 | 100 | — | |
| OmniVL# Img-Text Pairs=14M*, Fine-tuned=true2022.09 | 97.3 | 99.9 | 100 | — | |
| Florence#PT Data=0.9B2023.02 | 97.2 | 99.9 | — | — | |
| mPLUG-2#PT Data=17M2023.02 | 97.2 | 100 | 100 | — | |
| Florence# Img-Text Pairs=900M, Fine-tuned=true2022.09 | 97.2 | 99.9 | — | — | |
| X-VLM# Params=216M, # Pre-train Images=16M2021.11 | 97.1 | 100 | 100 | — | |
| mPLUG-2Base#PT Data=17M2023.02 | 96.9 | 100 | 100 | — | |
| X-VLM# Params=216M, # Pre-train Images=4M2021.11 | 96.8 | 99.8 | 100 | — | |
| X-VLMParameters=216M, Evaluation Protocol=Fine-Tuned, Training Data=Standard2023.01 | 96.8 | 99.8 | 100 | — | |
| BLIPPre-training data size=14M2022.06 | 96.6 | 99.8 | 100 | — | |
| BLIP# Img-Text Pairs=14M, Fine-tuned=true2022.09 | 96.6 | 99.8 | 100 | — | |
| SINGULARITYPre-training data size=17M2022.06 | 96.1 | 99.8 | 99.9 | — | |
| RaSa2023.05 | 96 | 99.8 | 100 | — | |
| ALBEFPre-training data size=14M2022.06 | 95.9 | 99.8 | 100 | — | |
| ALBEF# Params=210M, # Pre-train Images=14M2021.11 | 95.9 | 99.8 | 100 | — | |
| ALBEF#PT Data=14M2023.02 | 95.9 | 99.8 | 100 | — | |
| ALBEF# Img-Text Pairs=14M, Fine-tuned=true2022.09 | 95.9 | 99.8 | 100 | — | |
| METER + EPICData=4M, EPIC=true, Protocol=Fine-tuned2022.11 | 95.8 | 99.3 | 99.6 | — | |
| VL-BEITParameters=175M, Evaluation Protocol=Fine-Tuned, Training Data=Standard2023.01 | 95.8 | — | — | — | |
| ALBEFconfiguration=backbone2023.05 | 95.5 | 99.8 | 99.9 | — | |
| ALIGNPre-training data size=1.2B2022.06 | 95.3 | 99.8 | 100 | — | |
| ALIGN# Params=490M, # Pre-train Images=1.8B2021.11 | 95.3 | 99.8 | 100 | — | |
| VLMo#PT Data=4M2023.02 | 95.3 | 99.9 | 100 | — | |
| ALIGN#PT Data=1.8B2023.02 | 95.3 | 99.8 | 100 | — | |
| ALIGN2023.05 | 95.3 | 99.8 | 100 | — | |
| ALIGN# Img-Text Pairs=1.8B, Fine-tuned=true2022.09 | 95.3 | 99.8 | 100 | — | |
| OmniVLParameters=288M, Evaluation Protocol=Fine-Tuned, Training Data=Standard2023.01 | 94.9 | 99.6 | 99.9 | — | |
| BEIT-3Parameters=1.9B, Evaluation Protocol=Fine-Tuned, Training Data=Super-Large2023.01 | 94.9 | 99.9 | 100 | — | |
| OmniVL# Img-Text Pairs=4M*, Fine-tuned=true2022.09 | 94.9 | 99.6 | 99.9 | — | |
| ALBEFPre-training data size=4M2022.06 | 94.3 | 99.4 | 99.8 | — | |
| ALBEF# Params=210M, # Pre-train Images=4M2021.11 | 94.3 | 99.4 | 99.8 | — | |
| METER-CLIP# Params=380M, # Pre-train Images=4M2021.11 | 94.3 | 99.6 | 99.9 | — | |
| ALBEFParameters=210M, Evaluation Protocol=Fine-Tuned, Training Data=Standard2023.01 | 94.3 | 99.4 | 99.8 | — | |
| METER# Img-Text Pairs=404M, Fine-tuned=true2022.09 | 94.3 | 99.6 | 99.9 | — | |
| METERData=4M, EPIC=false, Protocol=Fine-tuned2022.11 | 94.2 | 99.6 | 99.9 | — | |
| X-FMbaseParameters=327M, Evaluation Protocol=Zero-Shot, Training Data=More Data2023.01 | 93.4 | 99.8 | 99.9 | — | |
| SINGULARITYPre-training data size=5M2022.06 | 93.3 | 99.4 | 99.8 | — | |
| METER + EPICData=4M, EPIC=true, Protocol=Zero-shot2022.11 | 93.1 | 99.1 | 99.8 | — | |
| CoCaParameters=2.1B, Evaluation Protocol=Zero-Shot, Training Data=Super-Large2023.01 | 92.5 | 99.5 | 99.9 | — | |
| METER-Swin# Params=380M, # Pre-train Images=4M2021.11 | 92.4 | 99 | 99.5 | — | |
| VLMbaseParameters=175M, Evaluation Protocol=Fine-Tuned, Training Data=Standard2023.01 | 92.3 | 99.4 | 99.9 | — | |
| VLMO# Img-Text Pairs=4M, Fine-tuned=true2022.09 | 92.3 | 99.4 | 99.9 | — | |
| METERData=4M, EPIC=false, Protocol=Zero-shot2022.11 | 92.2 | 99.3 | 99.7 | — | |
| FlorenceParameters=893M, Evaluation Protocol=Zero-Shot, Training Data=More Data2023.01 | 90.9 | 99.1 | — | — | |
| X-FMbaseParameters=327M, Evaluation Protocol=Zero-Shot, Training Data=Standard2023.01 | 90.1 | 99.2 | 99.9 | — | |
| UNIMO#PT Data=4M2023.02 | 89.4 | 98.9 | 99.8 | — | |
| UNIMO2023.05 | 89.4 | 98.9 | 99.8 | — | |
| UNIMO# Img-Text Pairs=4M, Fine-tuned=true2022.09 | 89.4 | 98.9 | 99.8 | — | |
| Knowledge-CLIPMode=Fine-tuned2022.10 | 89.2 | 98.9 | 99.4 | — | |
| COOKIE2023.05 | 89 | 98.9 | 99.7 | — | |
| ERNIE-ViLMode=Fine-tuned2022.10 | 88.7 | 98 | 99.2 | — | |
| ALIGN# Params=490M, # Pre-train Images=1.8B, Zero-shot=true2021.11 | 88.6 | 98.7 | 99.7 | — | |
| CLIPMode=Fine-tuned2022.10 | 88.6 | 98.5 | 99.4 | — | |
| ALIGNParameters=490M, Evaluation Protocol=Zero-Shot, Training Data=More Data2023.01 | 88.6 | 98.7 | 99.7 | — | |
| X2-VLMbaseParameters=255M, Evaluation Protocol=Zero-Shot, Training Data=More Data2023.01 | 88.6 | 98.7 | 98.9 | — | |
| CLIP# Params=~100M, # Pre-train Images=400M, Zero-shot=true2021.11 | 88 | 98.7 | 99.4 | — | |
| CLIPParameters=490M, Evaluation Protocol=Zero-Shot, Training Data=More Data2023.01 | 88 | 98.7 | 99.4 | — | |
| VILLAMode=Fine-tuned2022.10 | 87.9 | 97.5 | 98.8 | — | |
| Uni-PerceiverMode=Fine-tuned2022.10 | 87.9 | 98.2 | 99.1 | — | |
| X-VLM# Params=216M, # Pre-train Images=16M, Zero-shot=true2021.11 | 87.7 | 98.6 | 99.6 | — | |
| UNITERPre-training data size=4M2022.06 | 87.3 | 98 | 99.2 | — | |
| UNITER_large# Params=300M, # Pre-train Images=4M2021.11 | 87.3 | 98 | 99.2 | — | |
| UNITERMode=Fine-tuned2022.10 | 87.3 | 98 | 99.2 | — | |
| UNITER#PT Data=4M2023.02 | 87.3 | 98 | 99.2 | — | |
| UNITER2023.05 | 87.3 | 98 | 99.2 | — | |
| UNITER# Img-Text Pairs=4M, Fine-tuned=true2022.09 | 87.3 | 98 | 99.2 | — | |
| Unicoder-VLMode=Fine-tuned2022.10 | 86.2 | 96.3 | 99 | — | |
| E2E-VLP#PT Data=4M2023.02 | 86.2 | 97.5 | 98.92 | — | |
| FLAVA# Img-Text Pairs=70M, Fine-tuned=true2022.09 | 85.4 | 95.7 | 98.3 | — | |
| X-VLM# Params=216M, # Pre-train Images=4M, Zero-shot=true2021.11 | 85.3 | 97.8 | 99.6 | — | |
| X-VLMParameters=216M, Evaluation Protocol=Zero-Shot, Training Data=Standard2023.01 | 85.3 | 97.8 | 99.6 | — | |
| X2-VLMbaseParameters=255M, Evaluation Protocol=Zero-Shot, Training Data=Standard2023.01 | 85.1 | 99.2 | 100 | — | |
| ViLTPre-training data size=4M2022.06 | 83.5 | 96.7 | 98.6 | — | |
| ViLTMode=Fine-tuned2022.10 | 83.5 | 96.7 | 98.6 | — | |
| IN2RNoise=0.22026.06 | 80 | 95.5 | 97.7 | 511.6 | |
| SPS (IJCAI'24)Noise=0.22026.06 | 79.5 | 95 | 98 | 507.1 | |
| ESC (CVPR'24)Noise=0.22026.06 | 79 | 94.8 | 97.5 | 503.3 | |
| PCSR (AAAI'24)Noise=0.22026.06 | 78.7 | 95.1 | 97.9 | 505.7 | |
| GSC (CVPR'24)Noise=0.22026.06 | 78.3 | 94.6 | 97.8 | 505.8 | |
| IN2RNoise=0.42026.06 | 78.3 | 94.5 | 97.2 | 504.1 | |
| BiCro (CVPR'23)Noise=0.22026.06 | 78.1 | 94.4 | 97.5 | 504.7 | |
| L2RM (CVPR'24)Noise=0.22026.06 | 77.9 | 95.2 | 97.8 | 503.8 | |
| SPS (IJCAI'24)Noise=0.42026.06 | 77.8 | 93.6 | 97.1 | 498.9 | |
| CREAM (TIP'24)Noise=0.22026.06 | 77.4 | 95 | 97.5 | 502.3 | |
| PCSR (AAAI'24)Noise=0.42026.06 | 76.6 | 94.5 | 97.3 | 497.1 | |
| GSC (CVPR'24)Noise=0.42026.06 | 76.5 | 94.1 | 97.6 | 497.3 | |
| CREAM (TIP'24)Noise=0.42026.06 | 76.3 | 93.3 | 97.1 | 495 | |
| ESC (CVPR'24)Noise=0.42026.06 | 76.1 | 93.1 | 96.4 | 489.6 | |
| L2RM (CVPR'24)Noise=0.42026.06 | 75.8 | 93.2 | 96.9 | 490.5 | |
| NCR (NeurIPS'21)Noise=0.42026.06 | 75.3 | 92.1 | 95.2 | 486.8 | |
| IN2RNoise=0.62026.06 | 75.1 | 94 | 97.2 | 495.2 | |
| BiCro (CVPR'23)Noise=0.42026.06 | 74.6 | 92.7 | 96.2 | 487.5 |