Text-to-image Retrieval on Flickr30k (1K)
84.9R@1ALIGN
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ALIGN#Images=1.2B, Fine-tuning=true2022.02 | 84.9 | 97.4 | 98.6 | |
| Triple Contrastive Learning (TCL)#Images=4M, Fine-tuning=true2022.02 | 84 | 96.7 | 98.5 | |
| ALBEF#Images=4M, Fine-tuning=true2022.02 | 82.8 | 96.7 | 98.4 | |
| SoftMask++#Img=4M, Zero-shot=true2023.04 | 80.1 | 94.9 | 97.7 | |
| CODIS#Img=4M, Zero-shot=true2023.04 | 79.7 | 94.8 | 97.3 | |
| TCL#Images=4M, Zero-shot=true2022.02 | 79.6 | 95.1 | 97.4 | |
| TCL#Img=4M, Zero-shot=true2023.04 | 79.6 | 95.1 | 97.4 | |
| ALBEF#Images=4M, Zero-shot=true2022.02 | 76.8 | 93.7 | 96.7 | |
| ALBEF#Img=4M, Zero-shot=true2023.04 | 76.8 | 93.7 | 96.7 | |
| VILLA#Images=4M, Fine-tuning=true2022.02 | 76.3 | 94.2 | 96.8 | |
| VISTA-LTime (ms)=~40, Fine-tuning=true2022.03 | 75.8 | 94.2 | 96.9 | |
| ALIGN#Images=1.2B, Zero-shot=true2022.02 | 75.7 | 93.8 | 96.8 | |
| ALIGN#Img=1.2B, Zero-shot=true2023.04 | 75.7 | 93.8 | 96.8 | |
| UNITER#Images=4M, Fine-tuning=true2022.02 | 75.6 | 94.1 | 96.8 | |
| UNIMO#Images=4M, Fine-tuning=true2022.02 | 74.7 | 93.47 | 96.1 | |
| ERNIE-VIL-BTime (ms)=~920, Fine-tuning=true2022.03 | 74.4 | 92.7 | 95.9 | |
| VSEinftyFine-tuning=true2022.03 | 74.2 | 93.7 | 96.8 | |
| H Xue et al.Fine-tuning=true2022.03 | 73.5 | 93.1 | 96.4 | |
| ImageBERT#Images=6M, Fine-tuning=true2022.02 | 73.1 | 92.6 | 96 | |
| SOHO#Images=200K, Fine-tuning=true2022.02 | 72.5 | 92.7 | 96.1 | |
| UNITER-BTime (ms)=~900, Fine-tuning=true2022.03 | 72.5 | 92.4 | 96.1 | |
| SOHOFine-tuning=true2022.03 | 72.5 | 92.7 | 96.1 | |
| Miech et alFine-tuning=true2022.03 | 72.1 | 91.5 | 95.2 | |
| Unicoder-VLTime (ms)=925, Fine-tuning=true2022.03 | 71.5 | 91.2 | 95.2 | |
| Pixel-BERT-XTime (ms)=~160, Fine-tuning=true2022.03 | 71.5 | 92.1 | 95.8 | |
| VISTA-BTime (ms)=~17, Fine-tuning=true2022.03 | 68.9 | 91.1 | 95.1 | |
| CLIP#Images=400M, Zero-shot=true2022.02 | 68.7 | 90.6 | 95.2 | |
| CLIP#Img=400M, Zero-shot=true2023.04 | 68.7 | 90.6 | 95.2 | |
| 12-in-1Fine-tuning=true2022.03 | 67.9 | 89.6 | 94.2 | |
| UNITER-BaseVisual Embed=Region, Time (ms)=~900, Zero-Shot=true2021.02 | 66.2 | 88.4 | 92.9 | |
| UNITER#Images=4M, Zero-shot=true2022.02 | 66.2 | 88.4 | 92.9 | |
| UNITER#Img=4M, Zero-shot=true2023.04 | 66.2 | 88.4 | 92.9 | |
| ViLT#Images=4M, Fine-tuning=true2022.02 | 64.4 | 88.7 | 93.8 | |
| ViLT-BTime (ms)=~15, Fine-tuning=true2022.03 | 64.4 | 88.7 | 93.8 | |
| CLIP ViT-B/32|S|=400M2026.03 | 58.8 | 83.4 | 90 | |
| SGRAFFine-tuning=true2022.03 | 58.5 | 83 | 88.8 | |
| GSMNFine-tuning=true2022.03 | 57.4 | 82.3 | 89 | |
| ViLT-B/32Visual Embed=Linear, Time (ms)=~15, Zero-Shot=true, Pre-training steps=200K2021.02 | 55 | 82.5 | 89.8 | |
| ViLT#Images=4M, Zero-shot=true2022.02 | 55 | 82.5 | 89.8 | |
| ViLT#Img=4M, Zero-shot=true2023.04 | 55 | 82.5 | 89.8 | |
| VSRNFine-tuning=true2022.03 | 54.7 | 81.8 | 88.2 | |
| Procrustes|S|=5002026.03 | 54.4 | 69.5 | 78 | |
| ImageBERTVisual Embed=Region, Time (ms)=~925, Zero-Shot=true, Additional Pre-training Dataset=10M proprietary2021.02 | 54.3 | 79.6 | 87.5 | |
| ImageBERT#Images=6M, Zero-shot=true2022.02 | 54.3 | 79.6 | 87.5 | |
| IMRAMFine-tuning=true2022.03 | 53.9 | 79.4 | 87.2 | |
| Pixel-BERT-RTime (ms)=~60, Fine-tuning=true2022.03 | 53.4 | 80.4 | 88.5 | |
| ViLT-B/32Visual Embed=Linear, Time (ms)=~15, Zero-Shot=true2021.02 | 51.3 | 79.9 | 87.9 | |
| SCANFine-tuning=true2022.03 | 48.6 | 77.7 | 85.2 | |
| Unicoder-VLVisual Embed=Region, Time (ms)=~925, Zero-Shot=true2021.02 | 48.4 | 76 | 85.2 | |
| ViLBERTVisual Embed=Region, Time (ms)=~900, Zero-Shot=true2021.02 | 31.9 | 61.1 | 72.8 | |
| Relative Reps|S|=5002026.03 | 26.7 | 50.9 | 62.3 | |
| Relative Reps|S|=1002026.03 | 12.1 | 27.5 | 36.7 | |
| Procrustes|S|=1002026.03 | 10.8 | 23 | 32.9 | |
| FMap|S|=5002026.03 | 6.1 | 17.9 | 25.9 | |
| Relative Reps|S|=202026.03 | 3.5 | 9.3 | 14.1 | |
| Procrustes|S|=202026.03 | 2 | 5.7 | 10.4 | |
| FMap|S|=1002026.03 | 1.9 | 9.3 | 15 | |
| FMap|S|=202026.03 | 0.8 | 4.5 | 8.1 | |
| FMap Unsupervised (HKS)|S|=02026.03 | 0.4 | 1.7 | 3.2 | |
| Raw Cosine|S|=02026.03 | 0.1 | 0.5 | 1 | |
| CCA|S|=202026.03 | 0.1 | 0.4 | 1 | |
| CCA|S|=1002026.03 | 0.1 | 0.5 | 0.9 | |
| CCA|S|=5002026.03 | 0.1 | 0.4 | 0.9 |