Text Retrieval on MS-COCO 5K (test)
84.8R@1CoCa
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| CoCaParameters=2.1B, Evaluation Protocol=Fine-Tuned, Training Data=Super-Large2023.01 | 84.8 | 96.5 | 98.3 | |
| X-FMbaseParameters=327M, Evaluation Protocol=Fine-Tuned, Training Data=More Data2023.01 | 84.2 | 96.4 | 98.4 | |
| X2-VLMbaseParameters=255M, Evaluation Protocol=Fine-Tuned, Training Data=More Data2023.01 | 83.5 | 96.3 | 98.5 | |
| VLMO-Large++Pretrain Images=1.0B, Model Size Category=Large-Size, Interaction Protocol=Dual encoder (dot product), large batch size=true2021.11 | 83.1 | 96 | 98.2 | |
| mPLUG-2#PT Data=17M2023.02 | 82.5 | 95.7 | 98 | |
| BLIP#PT Data=129M2023.02 | 82.4 | 95.4 | 97.9 | |
| BLIP2023.05 | 82.4 | 95.4 | 97.9 | |
| OmniVL# Img-Text Pairs=14M*, Fine-tuned=true2022.09 | 82.1 | 95.9 | 98.1 | |
| BLIPPre-training data size=129M2022.06 | 81.9 | 95.4 | 97.8 | |
| Florence-HugePretrain Images=900M, Model Size Category=Huge-Size, Interaction Protocol=Dual encoder (dot product)2021.11 | 81.8 | 95.2 | — | |
| Florence#PT Data=0.9B2023.02 | 81.8 | 95.2 | — | |
| X-FMbaseParameters=327M, Evaluation Protocol=Fine-Tuned, Training Data=Standard2023.01 | 81.8 | 96 | 98.3 | |
| Florence# Img-Text Pairs=900M, Fine-tuned=true2022.09 | 81.8 | 95.2 | — | |
| BEIT-3Parameters=1.9B, Evaluation Protocol=Zero-Shot, Training Data=Super-Large2023.01 | 81.5 | 95.6 | 97.8 | |
| X-VLM# Params=216M, # Pre-train Images=16M2021.11 | 81.2 | 95.6 | 98.2 | |
| mPLUG-2Base#PT Data=17M2023.02 | 81.2 | 95.6 | 98.1 | |
| X-VLM + EPICData=4M+, EPIC=true, Protocol=Fine-tuned2022.11 | 81 | 95.3 | 97.9 | |
| X-VLM + EPICData=16M+, EPIC=true, Protocol=Fine-tuned2022.11 | 80.7 | 95.6 | 98 | |
| BLIPPre-training data size=14M2022.06 | 80.6 | 95.2 | 97.6 | |
| BLIP# Img-Text Pairs=14M, Fine-tuned=true2022.09 | 80.6 | 95.2 | 97.6 | |
| X2-VLMbaseParameters=255M, Evaluation Protocol=Fine-Tuned, Training Data=Standard2023.01 | 80.5 | 95.5 | 97.8 | |
| X-VLMParameters=216M, Evaluation Protocol=Fine-Tuned, Training Data=Standard2023.01 | 80.4 | 95.5 | 98.2 | |
| X-VLM# Params=216M, # Pre-train Images=4M2021.11 | 80.1 | 95.5 | 98.2 | |
| X-VLMData=4M+, EPIC=false, Protocol=Fine-tuned2022.11 | 79.8 | 95.1 | 97.7 | |
| METER + EPICData=16M, EPIC=true, Protocol=Fine-tuned2022.11 | 79.7 | 94.8 | 97.5 | |
| X-VLMData=16M+, EPIC=false, Protocol=Fine-tuned2022.11 | 79.5 | 95.4 | 97.8 | |
| VL-BEITParameters=175M, Evaluation Protocol=Fine-Tuned, Training Data=Standard2023.01 | 79.5 | — | — | |
| ALBEF + EPICData=16M, EPIC=true, Protocol=Fine-tuned2022.11 | 79.2 | 94.7 | 97.5 | |
| METER + EPICData=4M, EPIC=true, Protocol=Fine-tuned2022.11 | 79 | 94.5 | 97.5 | |
| ALBEFData=16M, EPIC=false, Protocol=Fine-tuned2022.11 | 78.3 | 93.9 | 96.8 | |
| VLMO-LargePretrain Images=4M, Model Size Category=Large-Size, Interaction Protocol=Dual encoder (dot product)2021.11 | 78.2 | 94.4 | 97.4 | |
| VLMo#PT Data=4M2023.02 | 78.2 | 94.4 | 97.4 | |
| METERData=16M, EPIC=false, Protocol=Fine-tuned2022.11 | 78.2 | 93.8 | 96.9 | |
| CCLM_basePre-training Data=4M2022.06 | 77.72 | — | — | |
| ALBEFPre-train Images=14M, Fine-tuned=true2021.07 | 77.6 | 94.3 | 97.2 | |
| ALBEFPre-training data size=14M2022.06 | 77.6 | 94.3 | 97.2 | |
| ALBEF# Params=210M, # Pre-train Images=14M2021.11 | 77.6 | 94.3 | 97.2 | |
| ALBEF#PT Data=14M2023.02 | 77.6 | 94.3 | 97.2 | |
| X-FMbaseParameters=327M, Evaluation Protocol=Zero-Shot, Training Data=More Data2023.01 | 77.6 | 94.8 | 97.7 | |
| ALBEF# Img-Text Pairs=14M, Fine-tuned=true2022.09 | 77.6 | 94.3 | 97.2 | |
| RaSa2023.05 | 77.44 | 94.12 | 97.18 | |
| AlignCMSS2023.09 | 77.4 | 94.1 | 97 | |
| ALBEFconfiguration=backbone2023.05 | 77.26 | 94.02 | 97.04 | |
| METERData=4M, EPIC=false, Protocol=Fine-tuned2022.11 | 77.2 | 93.7 | 97.1 | |
| ALIGNPre-train Images=1.2B, Fine-tuned=true2021.07 | 77 | 93.5 | 96.9 | |
| ALIGN-LargePretrain Images=1.8B, Model Size Category=Large-Size, Interaction Protocol=Dual encoder (dot product)2021.11 | 77 | 93.5 | 96.9 | |
| ALIGNPre-training data size=1.2B2022.06 | 77 | 93.5 | 96.9 | |
| SINGULARITYPre-training data size=17M2022.06 | 77 | 93.7 | 96.8 | |
| ALIGN# Params=490M, # Pre-train Images=1.8B2021.11 | 77 | 93.5 | 96.9 | |
| ALIGNTraining Data=1.8B2022.07 | 77 | 93.5 | 96.9 | |
| ALIGN#PT Data=1.8B2023.02 | 77 | 93.5 | 96.9 | |
| ALIGN2023.05 | 77 | 93.5 | 96.9 | |
| ALIGN# Img-Text Pairs=1.8B, Fine-tuned=true2022.09 | 77 | 93.5 | 96.9 | |
| OmniVLParameters=288M, Evaluation Protocol=Fine-Tuned, Training Data=Standard2023.01 | 76.8 | 93.6 | 97.3 | |
| OmniVL# Img-Text Pairs=4M*, Fine-tuned=true2022.09 | 76.8 | 93.6 | 97.3 | |
| METER-CLIP# Params=380M, # Pre-train Images=4M2021.11 | 76.2 | 93.2 | 96.8 | |
| METER# Img-Text Pairs=404M, Fine-tuned=true2022.09 | 76.2 | 93.2 | 96.8 | |
| OSCAR+ w/ VINVLBERT scale=Large2021.01 | 75.4 | 92.9 | 96.2 | |
| VinVL-LargePretrain Images=5.7M, Model Size Category=Large-Size, Interaction Protocol=Fusion encoder2021.11 | 75.4 | 92.9 | 96.2 | |
| VinVL_large# Params=550M, # Pre-train Images=5.6M2021.11 | 75.4 | 92.9 | 96.2 | |
| ALBEF + EPICData=4M, EPIC=true, Protocol=Fine-tuned2022.11 | 75.1 | 92.9 | 96.4 | |
| VLMO-BasePretrain Images=4M, Model Size Category=Base-Size, Interaction Protocol=Dual encoder (dot product)2021.11 | 74.8 | 93.1 | 96.9 | |
| VLMbaseParameters=175M, Evaluation Protocol=Fine-Tuned, Training Data=Standard2023.01 | 74.8 | 93.1 | 96.9 | |
| VLMO# Img-Text Pairs=4M, Fine-tuned=true2022.09 | 74.8 | 93.1 | 96.9 | |
| OSCAR+ w/ VINVLBERT scale=Base2021.01 | 74.6 | 92.6 | 96.3 | |
| VinVL-BaseVisual Embed=Region, Time (ms)=~650, Extra pre-training=GQA, VQAv2, VG-QA, Extra pre-training dataset=Open Images2021.02 | 74.6 | 92.6 | 96.3 | |
| VinVL (Base)Training Data=8.9M2022.07 | 74.6 | 92.6 | 96.3 | |
| VinVL_baseModel Scale=base2022.06 | 74.6 | — | — | |
| VinVL2023.09 | 74.6 | 92.6 | 96.3 | |
| X-FMbaseParameters=327M, Evaluation Protocol=Zero-Shot, Training Data=Standard2023.01 | 73.8 | 93.9 | 97.2 | |
| OSCARModel Size=Large2020.04 | 73.5 | 92.2 | 96 | |
| OSCARSize=L2020.04 | 73.5 | 92.2 | 96 | |
| OscarEvaluation Protocol=Finetune2021.02 | 73.5 | 92.2 | 96 | |
| OscarBERT scale=Large2021.01 | 73.5 | 92.2 | 96 | |
| OSCARMode=Fine-tuned2022.10 | 73.5 | 92.2 | 96 | |
| Oscar2023.05 | 73.5 | 92.2 | 96 | |
| ALBEFData=4M, EPIC=false, Protocol=Fine-tuned2022.11 | 73.3 | 92.4 | 96.4 | |
| X-VLMData=16M+, EPIC=true, Evaluation Protocol=zero-shot2022.11 | 73.2 | 93.6 | 96.9 | |
| X-VLM + EPICData=16M+, EPIC=true, Protocol=Zero-shot2022.11 | 73.2 | 93.6 | 96.9 | |
| ALBEFPre-train Images=4M, Fine-tuned=true2021.07 | 73.1 | 91.4 | 96 | |
| ALBEF-BasePretrain Images=4M, Model Size Category=Base-Size, Interaction Protocol=Fusion reranking2021.11 | 73.1 | 91.4 | 96 | |
| ALBEFPre-training data size=4M2022.06 | 73.1 | 91.4 | 96 | |
| ALBEF# Params=210M, # Pre-train Images=4M2021.11 | 73.1 | 91.4 | 96 | |
| ALBEFPre-training Data=4M2022.06 | 73.1 | — | — | |
| X-VLMData=16M+, EPIC=false, Evaluation Protocol=zero-shot2022.11 | 73.1 | 92.8 | 97 | |
| X-VLMData=16M+, EPIC=false, Protocol=Zero-shot2022.11 | 73.1 | 92.8 | 97 | |
| ALBEFParameters=210M, Evaluation Protocol=Fine-Tuned, Training Data=Standard2023.01 | 73.1 | 91.4 | 96 | |
| METER-Swin# Params=380M, # Pre-train Images=4M2021.11 | 73 | 92 | 96.3 | |
| X-VLMData=4M+, EPIC=true, Evaluation Protocol=zero-shot2022.11 | 72 | 93.4 | 97.3 | |
| X-VLM + EPICData=4M+, EPIC=true, Protocol=Zero-shot2022.11 | 72 | 93.4 | 97.3 | |
| SINGULARITYPre-training data size=5M2022.06 | 71.9 | 90.8 | 95.4 | |
| X2-VLMbaseParameters=255M, Evaluation Protocol=Zero-Shot, Training Data=More Data2023.01 | 71.7 | 93.4 | 97.5 | |
| X-VLM# Params=216M, # Pre-train Images=16M, Zero-shot=true2021.11 | 71.6 | 93.1 | 97 | |
| COOKIE2023.05 | 71.6 | 90.9 | 95.4 | |
| X-VLM# Params=216M, # Pre-train Images=4M, Zero-shot=true2021.11 | 70.8 | 92.1 | 96.5 | |
| X-VLMParameters=216M, Evaluation Protocol=Zero-Shot, Training Data=Standard2023.01 | 70.8 | 92.1 | 96.5 | |
| Knowledge-CLIPMode=Fine-tuned2022.10 | 70.2 | 89.2 | 94.4 | |
| OSCARModel Size=Base2020.04 | 70 | 91.1 | 95.5 | |
| OSCARSize=B2020.04 | 70 | 91.1 | 95.5 | |
| OscarBERT scale=Base2021.01 | 70 | 91.1 | 95.5 |