Text-to-Image Retrieval on MSCOCO 1K (test)
6,390R@1AltCLIP
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| AltCLIPBackbone=ViT-L, Zero-shot=true, Stage=Contrastive Learning2022.11 | 6,390 | 8,720 | 9,390 | 82 | — | — | — | |
| CN-CLIPBackbone=ViT-L, Zero-shot=true2022.11 | 6,370 | 8,870 | 9,440 | 81 | — | — | — | |
| R2D2Backbone=ViT-L, Zero-shot=true, Reported original results=true2022.11 | 6,330 | 8,930 | 9,570 | 80.5 | — | — | — | |
| AltCLIPTBackbone=ViT-L, Zero-shot=true, Stage=Teacher Learning2022.11 | 5,570 | 8,290 | 9,120 | 78.4 | — | — | — | |
| WukongBackbone=ViT-L, Zero-shot=true2022.11 | 5,520 | 8,100 | 9,060 | 75.1 | — | — | — | |
| TaiyiBackbone=ViT-L, Zero-shot=true2022.11 | 5,200 | 8,020 | 8,960 | 72.2 | — | — | — | |
| OscarQuery dependence=Query-dependent2021.01 | 75.7 | 95.2 | 98.3 | — | — | — | — | |
| SOHOBackbone=R1012021.04 | 73.5 | 94.5 | 97.5 | — | — | — | — | |
| Unicoder-VL2021.04 | 69.7 | 93.5 | 97.2 | — | — | — | — | |
| Unicoder-VLQuery dependence=Query-dependent2021.01 | 69.7 | 93.5 | 97.2 | — | — | — | — | |
| VisualSpartaQuery dependence=Query-agnostic2021.01 | 68.7 | 91.2 | 96.2 | — | — | — | — | |
| CMSFBackbone Architecture=Faster R-CNN + BERT, Ensemble=false2026.03 | 67.1 | 92.4 | 96.8 | — | — | — | 533.8 | |
| TERANBackbone Architecture=Faster R-CNN + BERT, Ensemble=true2026.03 | 67 | 92.2 | 96.9 | — | — | — | 531.9 | |
| CHANBackbone Architecture=Faster R-CNN + BERT, Ensemble=false2026.03 | 66.5 | 92.1 | 96.7 | — | — | — | 532.6 | |
| MaxMatchBackbone Architecture=Faster R-CNN + BERT, Ensemble=false2026.03 | 66.4 | 91.9 | 96.6 | — | — | — | 533.8 | |
| HREMBackbone Architecture=Faster R-CNN + BERT, Ensemble=false2026.03 | 66.1 | 91.6 | 96.5 | — | — | — | 530.7 | |
| USERBackbone Architecture=Faster R-CNN + BERT, Ensemble=false2026.03 | 66.1 | 90.6 | 95.6 | — | — | — | 530.5 | |
| VSE∞Backbone Architecture=Faster R-CNN + BERT, Ensemble=false2026.03 | 64.8 | 91.4 | 96.3 | — | — | — | 527.5 | |
| VSRN++Backbone Architecture=Faster R-CNN + BERT, Ensemble=true2026.03 | 64.1 | 91 | 96.1 | — | — | — | 523.6 | |
| CMSFBackbone Architecture=Faster R-CNN + BiGRU, Ensemble=false2026.03 | 63.6 | 91 | 96.3 | — | — | — | 524.7 | |
| NAAFBackbone Architecture=Faster R-CNN + BiGRU, Ensemble=false2026.03 | 63.5 | 89.6 | 95.3 | — | — | — | 521.2 | |
| VSRNBackbone Architecture=Faster R-CNN + BiGRU, Ensemble=true2026.03 | 62.8 | 89.7 | 95.1 | — | — | — | 516.8 | |
| VSE∞Backbone Architecture=Faster R-CNN + BiGRU, Ensemble=false2026.03 | 61.7 | 90.3 | 95.6 | — | — | — | 520.8 | |
| PFANQuery dependence=Query-agnostic2021.01 | 61.6 | 89.6 | 95.2 | — | — | — | — | |
| MMCABackbone Architecture=Faster R-CNN + BERT, Ensemble=false2026.03 | 61.6 | 89.8 | 95.2 | — | — | — | 514.7 | |
| CLIPBackbone=ViT-L, Zero-shot=true2022.11 | 60 | 410 | 710 | 5.4 | — | — | — | |
| CVSEQuery dependence=Query-agnostic2021.01 | 59.9 | 89.4 | 95.2 | — | — | — | — | |
| SCANBackbone=Faster R-CNN, ResNet, Retrieval direction=t-i + i-t ensemble, Pooling type=LSE + AVG2018.03 | 58.8 | 88.4 | 94.8 | — | — | — | — | |
| SCANBackbone=R1012021.04 | 58.8 | 88.4 | 94.8 | — | — | — | — | |
| SCANQuery dependence=Query-agnostic2021.01 | 58.8 | 88.4 | 94.8 | — | — | — | — | |
| SCANBackbone Architecture=Faster R-CNN + BiGRU, Ensemble=true2026.03 | 58.8 | 88.4 | 94.8 | — | — | — | 507.9 | |
| CAMPQuery dependence=Query-agnostic2021.01 | 58.5 | 87.9 | 95 | — | — | — | — | |
| SCOBackbone=ResNet2018.03 | 56.7 | 87.5 | 94.8 | — | — | — | — | |
| semantic-enhanced image and sentence matching modelBackbone=ResNet-1522017.12 | 56.7 | 87.5 | 94.8 | 83.2 | — | — | — | |
| SCO2019.06 | 56.7 | 87.5 | 94.8 | — | — | — | — | |
| GXNBackbone=ResNet2018.03 | 56.6 | — | 94.5 | — | — | — | — | |
| GXN2019.06 | 56.6 | — | 94.5 | — | — | — | — | |
| SCANBackbone=Faster R-CNN, ResNet, Retrieval direction=Text-Image (t-i), Pooling type=AVG, lambda1=92018.03 | 56.4 | 87 | 93.9 | — | — | — | — | |
| semantic-enhanced image and sentence matching modelBackbone=VGGNet-192017.12 | 55.5 | 86.6 | 93.8 | 81.8 | — | — | — | |
| PVSENumber of embeddings (K)=32019.06 | 55.2 | 86.5 | 93.7 | — | — | — | — | |
| SCANBackbone=Faster R-CNN, ResNet, Retrieval direction=Image-Text (i-t), Pooling type=LSE, lambda1=4, lambda2=202018.03 | 54.8 | 86.1 | 93.3 | — | — | — | — | |
| SCANBackbone=Faster R-CNN, ResNet, Retrieval direction=Image-Text (i-t), Pooling type=AVG, lambda1=42018.03 | 54.4 | 86 | 93.6 | — | — | — | — | |
| PVSENumber of embeddings (K)=12019.06 | 53.5 | 85.1 | 92.7 | — | — | — | — | |
| SCANBackbone=Faster R-CNN, ResNet, Retrieval direction=Text-Image (t-i), Pooling type=LSE, lambda1=9, lambda2=62018.03 | 53 | 85.4 | 92.9 | — | — | — | — | |
| VSE++Backbone=ResNet2018.03 | 52 | — | 92 | — | — | — | — | |
| VSE++Backbone=ResNet-1522017.12 | 52 | 83.1 | 92 | 79.4 | — | — | — | |
| VSE++2019.06 | 52 | 84.3 | 92 | — | — | — | — | |
| VSE++Backbone=R1522021.04 | 52 | 84.3 | 92 | — | — | — | — | |
| VSE++Query dependence=Query-agnostic2021.01 | 52 | — | 92 | — | — | — | — | |
| DPCBackbone=ResNet2018.03 | 47.1 | 79.9 | 90 | — | — | — | — | |
| Ours (ResNet-50) Stage IIVisual=ft ResNet-50, Textual=ft ResNet-50 (w2v init.)2017.11 | 47.1 | 79.9 | 90 | 2 | — | — | — | |
| VSE++Backbone=VGGNet-192017.12 | 45.9 | 78.9 | 89.1 | 74.6 | — | — | — | |
| Multimodal Contrastive TrainingBackbone=ResNet-50, Pre-training=Vision-and-Language (COCO)2021.04 | 45.1 | — | 90 | 2 | — | — | — | |
| CMPM2019.06 | 44.6 | 78.8 | 89 | — | — | — | — | |
| VirTexBackbone=ResNet-50, Pre-training=Vision-and-Language (COCO)2021.04 | 44 | — | 88.5 | 2 | — | — | — | |
| RRFBackbone=ResNet-1522017.12 | 43.9 | 78.1 | 88.6 | 73.9 | — | — | — | |
| RRF-Net2019.06 | 43.9 | 78.1 | 88.6 | — | — | — | — | |
| RRF-NetVisual=fixed ResNet-152, Textual=w2v + HGLMM2017.11 | 43.9 | 78.1 | 88.6 | — | — | — | — | |
| IN-supBackbone=ResNet-50, Pre-training=Supervised (ImageNet-1K)2021.04 | 42.8 | — | 87 | 2 | — | — | — | |
| Ours (VGG-19) Stage IIVisual=ft VGG-19, Textual=ft ResNet-50 (w2v init.)2017.11 | 41.6 | 76.3 | 87.5 | 2 | — | — | — | |
| SM-LSTMBackbone=VGG2018.03 | 40.7 | 75.8 | 87.4 | — | — | — | — | |
| sm-LSTMBackbone=VGGNet-192017.12 | 40.7 | 75.8 | 87.4 | 72 | — | — | — | |
| SM-LSTMQuery dependence=Query-agnostic2021.01 | 40.7 | 75.8 | 87.4 | — | — | — | — | |
| sm-LSTMVisual=fixed VGG-19, Textual=ft RNN2017.11 | 40.7 | 75.8 | 87.4 | 2 | — | — | — | |
| 2WayNetBackbone=VGG2018.03 | 39.7 | 63.3 | — | — | — | — | — | |
| 2WayNetBackbone=VGGNet-192017.12 | 39.7 | 63.3 | — | — | — | — | — | |
| 2WayNet2019.06 | 39.7 | 63.3 | — | — | — | — | — | |
| 2WayNetVisual=fixed VGG-16, Textual=feature from [17]2017.11 | 39.7 | 63.3 | — | — | — | — | — | |
| DSPEBackbone=VGGNet-192017.12 | 39.6 | 75.2 | 86.9 | 70.1 | — | — | — | |
| DSPE2019.06 | 39.6 | 75.2 | 86.9 | — | — | — | — | |
| SPEVisual=fixed VGG-19, Textual=w2v + HGLMM2017.11 | 39.6 | 75.2 | 86.9 | — | — | — | — | |
| MoCo-v2Backbone=ResNet-50, Pre-training=Self-supervised (MoCo-v2)2021.04 | 39 | — | 84.8 | 2 | — | — | — | |
| Order-embeddingsSupervision type=Direct supervision2017.05 | 37.9 | — | 85.9 | 2 | — | — | — | |
| Order-embeddingsBackbone=VGG2018.03 | 37.9 | — | 85.9 | — | — | — | — | |
| order-embeddings2015.11 | 37.9 | — | 85.9 | 2 | 8.1 | — | — | |
| Order-embeddingsTraining Data=COCO2016.09 | 37.9 | — | 85.9 | 2 | — | — | — | |
| OEMBackbone=VGGNet-192017.12 | 37.9 | 73.7 | 85.9 | 68.6 | — | — | — | |
| Order2019.06 | 37.9 | — | 85.9 | — | — | — | — | |
| M2 (LSTM + SA-FC7)Training Data=COCO2016.09 | 37.2 | 72.3 | 85.9 | 2 | — | — | — | |
| Ours (ResNet-50) Stage IVisual=fixed ResNet-50, Textual=ft ResNet-50 (w2v init.)2017.11 | 37.2 | 69.5 | 80.6 | 2 | — | — | — | |
| VQABackbone=VGGNet-192017.12 | 37 | 70.9 | 82.9 | 68.5 | — | — | — | |
| VQA-A2019.06 | 37 | 70.9 | 82.9 | — | — | — | — | |
| VQA-AVisual=fixed VGG-19, Textual=ft RNN2017.11 | 37 | 70.9 | 82.9 | — | — | — | — | |
| order-embeddingsVariant=symmetric2015.11 | 36.3 | — | 85.8 | 2 | 9 | — | — | |
| HM-LSTMBackbone=R-CNN, AlexNet2018.03 | 36.1 | — | 86.7 | — | — | — | — | |
| HM-LSTMVisual=fixed CNN from [32], Textual=ft RNN2017.11 | 36.1 | — | 86.7 | 3 | — | — | — | |
| Ours (VGG-19) Stage IVisual=fixed VGG-19, Textual=ft ResNet-50 (w2v init.)2017.11 | 34.4 | 66.6 | 78.7 | 3 | — | — | — | |
| BiLSTM-Max (on AllNLI)Supervision type=Pre-trained, Backbone=ResNet101, Training samples count=113k, Pre-training dataset=AllNLI2017.05 | 33.9 | 69.7 | 83.8 | 3 | — | — | — | |
| order-embeddingsFeatures=1-crop VGG2015.11 | 33.5 | — | 82.2 | 2.6 | 10 | — | — | |
| BiLSTM-Max (on SNLI)Supervision type=Pre-trained, Backbone=ResNet101, Training samples count=113k, Pre-training dataset=SNLI2017.05 | 33.2 | 69.7 | 83.6 | 3 | — | — | — | |
| m-CNNENSSupervision type=Direct supervision2017.05 | 32.6 | — | 82.8 | 3 | — | — | — | |
| m-CNN ENSEnsemble=true2015.11 | 32.6 | — | 82.8 | 3 | — | — | — | |
| m-CNNBackbone=VGGNet-192017.12 | 32.6 | 68.6 | 82.8 | 64 | — | — | — | |
| m-CNN2019.06 | 32.6 | 68.6 | 82.8 | — | — | — | — | |
| m-CNNVisual=ft VGG-19, Textual=4 × ft CNN2017.11 | 32.6 | 68.6 | 82.8 | 3 | — | — | — | |
| MNLM2015.11 | 31 | — | 79.9 | 3 | — | — | — | |
| MNLMBackbone=VGGNet-192017.12 | 31 | 66.7 | 79.9 | 63.8 | — | — | — | |
| SkipThoughtSupervision type=Pre-trained, Backbone=ResNet101, Training samples count=113k2017.05 | 30.6 | 66.2 | 81 | 3 | — | — | — | |
| RNN+FVBackbone=VGGNet-192017.12 | 29.6 | 64.8 | 80.5 | 61.8 | — | — | — | |
| RNN-FVVisual=fixed VGG-19, Textual=feature from [17]2017.11 | 29.2 | 64.7 | 80.4 | 3 | — | — | — |