Image Retrieval on MS-COCO 1K (test)
80.1R@1AlignCMSS
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| AlignCMSS2023.09 | 80.1 | 97.3 | 99.2 | — | — | |
| OSCAR+ w/ VINVLBERT scale=Large2021.01 | 78.8 | 96.1 | 98.5 | — | — | |
| OSCARModel Size=Large2020.04 | 78.2 | 95.8 | 98.3 | — | — | |
| OSCARSize=L2020.04 | 78.2 | 95.8 | 98.3 | — | — | |
| OSCAR+ w/ VINVLBERT scale=Base2021.01 | 78.2 | 95.6 | 98 | — | — | |
| VinVL2023.09 | 78.2 | 95.6 | 98 | — | — | |
| OSCARModel Size=Base2020.04 | 75.7 | 95.2 | 98.3 | — | — | |
| OSCARSize=B2020.04 | 75.7 | 95.2 | 98.3 | — | — | |
| OscarBERT scale=Base2021.01 | 75.7 | 95.2 | 98.3 | — | — | |
| ALADIN A/ft.Training Data=8.9M, head=alignment2022.07 | 75.4 | 95.2 | 97.9 | — | — | |
| ALADIN A/ft. + D/ft.Training Data=8.9M, head=alignment, distillation=true2022.07 | 75 | 95.2 | 98 | — | — | |
| ImageBERTtraining_mode=fine-tuned2020.01 | 73.6 | 94.3 | 97.2 | — | — | |
| ALADIN A/ft. + D/ft.Training Data=8.9M2022.07 | 72.7 | 94.5 | 97.5 | — | — | |
| ALADIN T/ft.Training Data=8.9M2022.07 | 71.9 | 93.8 | 97 | — | — | |
| ALADIN DTraining Data=8.9M2022.07 | 70.5 | 93.6 | 97.3 | — | — | |
| Unicoder-VLEvaluation Protocol=pre-train + fine-tune2019.08 | 69.7 | 93.5 | 97.2 | — | — | |
| Unicoder-VLtraining_mode=fine-tuned2020.01 | 69.7 | 93.5 | 97.2 | — | — | |
| Unicoder-VLModel Size=Base2020.04 | 69.7 | 93.5 | 97.2 | — | — | |
| Unicoder-VLSize=B2020.04 | 69.7 | 93.5 | 97.2 | — | — | |
| Unicoder-VLBERT scale=Base2021.01 | 69.7 | 93.5 | 97.2 | — | — | |
| Unicoder-VLTraining Data=3.8M2022.07 | 69.7 | 93.5 | 97.2 | — | — | |
| ALADIN TTraining Data=8.9M2022.07 | 68.9 | 92.8 | 96.6 | — | — | |
| SOTA [23]2019.12 | 68.5 | — | — | — | — | |
| Our SAT->STBackbone=BERTB, Training Strategy=All-Task Pretraining + Single-Task Finetuning, Number of Parameters=3B, Number of Models=12 x 250M2019.12 | 68 | — | — | — | — | |
| TERAN MrSwBackbone=Region CNN, Ensemble=true2020.08 | 67 | 92.2 | 96.9 | 0.747 | 0.68 | |
| TERANTraining Data=0.6M, Ensemble=true2022.07 | 67 | 92.2 | 96.9 | — | — | |
| 12-in-1Model Size=Base2020.04 | 65.2 | 91 | 96.2 | — | — | |
| 12-in-1Size=B2020.04 | 65.2 | 91 | 96.2 | — | — | |
| 12-in-1Training Data=4.4M2022.07 | 65.2 | 91 | 96.2 | — | — | |
| 12-in-1training=multi-task2019.12 | 65.16 | 91 | 96.2 | — | — | |
| TERAN MrSwBackbone=Region CNN2020.08 | 65 | 91.2 | 96.4 | 0.741 | 0.668 | |
| DSRANTraining Data=0.6M, Backbone=BERT2022.07 | 64.5 | 90.8 | 95.8 | — | — | |
| Unicoder-VLEvaluation Protocol=w/o pre-training2019.08 | 63.9 | 91.6 | 96.5 | — | — | |
| Discrete-continuous PGNetwork=Faster R-CNN2021.04 | 63.9 | 88.9 | 95.6 | — | — | |
| Our SATBackbone=BERTB, Training Strategy=All-Task (AT), Number of Parameters=270M, Number of Models=1 x 270M2019.12 | 63.7 | — | — | — | — | |
| TERAN Symm.Backbone=Region CNN2020.08 | 63.5 | 91.1 | 96.3 | 0.739 | 0.666 | |
| CAMERABackbone=Region CNN, BERT language model=true, Disentangled visual-textual pipelines=true, Ensemble=true2020.08 | 63.4 | 90.9 | 95.8 | — | — | |
| CAMERATraining Data=0.6M, Ensemble=true2022.07 | 63.4 | 90.9 | 95.8 | — | — | |
| SGRAFTraining strategy=independent training2021.01 | 63.2 | 90.7 | 96.1 | — | — | |
| VSRNBackbone=Faster R-CNN, ResNet2019.09 | 62.8 | 89.7 | 95.1 | — | — | |
| VSRN2021.01 | 62.8 | 89.7 | 95.1 | — | — | |
| VSRNBackbone=Region CNN, Disentangled visual-textual pipelines=true, Ensemble=true2020.08 | 62.8 | 89.7 | 95.1 | 0.732 | 0.637 | |
| VSRNNetwork=Faster R-CNN2021.04 | 62.8 | 89.7 | 95.1 | — | — | |
| DPRNN2021.01 | 62.5 | 89.7 | 95.1 | — | — | |
| CAMERABackbone=Region CNN, BERT language model=true, Disentangled visual-textual pipelines=true2020.08 | 62.3 | 90.1 | 95.2 | — | — | |
| SAF2021.01 | 61.8 | 89.4 | 95.3 | — | — | |
| IMRAM2021.01 | 61.7 | 89.1 | 95 | — | — | |
| Full-IMRAMBackbone=Region CNN2020.08 | 61.7 | 89.1 | 95 | — | — | |
| PFAN2019.08 | 61.6 | 89.6 | 95.2 | — | — | |
| PFANtraining_mode=fine-tuned2020.01 | 61.6 | 89.6 | 95.2 | — | — | |
| PFAN2020.04 | 61.6 | 89.6 | 95.2 | — | — | |
| PFAN2020.04 | 61.6 | 89.6 | 95.2 | — | — | |
| RDAN2021.01 | 61.6 | 89.2 | 94.7 | — | — | |
| MMCA2021.01 | 61.6 | 89.8 | 95.2 | — | — | |
| PFAN2021.01 | 61.6 | 89.6 | 95.2 | — | — | |
| MMCABackbone=Region CNN, BERT language model=true2020.08 | 61.6 | 89.8 | 95.2 | — | — | |
| PFANBackbone=Region CNN, Ensemble=true2020.08 | 61.6 | 89.6 | 95.2 | — | — | |
| SCG2019.08 | 61.4 | 88.9 | 95.1 | — | — | |
| SCGtraining_mode=fine-tuned2020.01 | 61.4 | 88.9 | 95.1 | — | — | |
| SCG2020.04 | 61.4 | 88.9 | 95.1 | — | — | |
| SCG2020.04 | 61.4 | 88.9 | 95.1 | — | — | |
| SGR2021.01 | 61.4 | 89.3 | 95.4 | — | — | |
| CAAN2021.01 | 61.3 | 89.7 | 95.2 | — | — | |
| VSRN*Model Type=single model2021.01 | 60.8 | 88.4 | 94.1 | — | — | |
| SANBackbone=VGG2020.08 | 60.8 | 90.3 | 95.7 | — | — | |
| VSRNBackbone=Region CNN, Disentangled visual-textual pipelines=true2020.08 | 60.8 | 88.4 | 94.1 | 0.723 | 0.621 | |
| AAMELBackbone=ResNet2020.08 | 59.9 | 89 | 95.1 | — | — | |
| BFAN2021.01 | 59.4 | 88.4 | — | — | — | |
| CASCBackbone=Region CNN2020.08 | 58.9 | 89.8 | 96 | — | — | |
| SCANBackbone=Faster R-CNN, ResNet2019.09 | 58.8 | 88.4 | 94.8 | — | — | |
| SCAN2019.08 | 58.8 | 88.4 | 94.8 | — | — | |
| SCANtraining_mode=fine-tuned2020.01 | 58.8 | 88.4 | 94.8 | — | — | |
| SCAN2020.04 | 58.8 | 88.4 | 94.8 | — | — | |
| SCAN2020.04 | 58.8 | 88.4 | 94.8 | — | — | |
| SCAN2021.01 | 58.8 | 88.4 | 94.8 | — | — | |
| SMANBackbone=ResNet2020.08 | 58.8 | 87.4 | 93.5 | — | — | |
| SCANBackbone=Region CNN, Ensemble=true2020.08 | 58.8 | 88.4 | 94.8 | — | — | |
| SCANNetwork=Faster R-CNN2021.04 | 58.8 | 88.4 | 94.8 | — | — | |
| CAMP2020.04 | 58.5 | 87.9 | 95 | — | — | |
| CAMP2020.04 | 58.5 | 87.9 | 95 | — | — | |
| CAMP2021.01 | 58.5 | 87.9 | 95 | — | — | |
| M3A-NetBackbone=ResNet2020.08 | 58.4 | 87.1 | 94 | — | — | |
| VilBERTTraining Data=3.1M2022.07 | 58.2 | 84.9 | 91.5 | — | — | |
| SAEMBackbone=Region CNN, BERT language model=true, Disentangled visual-textual pipelines=true, Ensemble=true2020.08 | 57.8 | 88.6 | 94.9 | — | — | |
| SAEMTraining Data=0.6M, Ensemble=true2022.07 | 57.8 | 88.6 | 94.9 | — | — | |
| SGM2021.01 | 57.5 | 87.3 | 94.3 | — | — | |
| TERAN MwSrBackbone=Region CNN2020.08 | 57.5 | 88.4 | 94.9 | 0.73 | 0.658 | |
| HDCtraining=multi-task2019.12 | 57.4 | 88.4 | 95.6 | — | — | |
| SCOBackbone=ResNet2019.09 | 56.7 | 87.5 | 94.8 | — | — | |
| SCONetwork=ResNet2021.04 | 56.7 | 87.5 | 94.3 | — | — | |
| GXNBackbone=ResNet2019.09 | 56.6 | — | 94.5 | — | — | |
| GXNNetwork=ResNet2021.04 | 56.6 | — | 94.5 | — | — | |
| TERN w. AlignBackbone=Region CNN2020.08 | 54.5 | 86.9 | 94.2 | 0.724 | 0.643 | |
| TERNTraining Data=0.6M2022.07 | 54.5 | 86.9 | 94.2 | — | — | |
| VSE++Backbone=ResNet2019.09 | 52 | 83.1 | 92 | — | — | |
| VSE++2019.08 | 52 | — | 92 | — | — | |
| VSE++2020.04 | 52 | — | 92 | — | — | |
| VSE++2020.04 | 52 | — | 92 | — | — | |
| VSE++Backbone=ResNet, Disentangled visual-textual pipelines=true2020.08 | 52 | 84.3 | 92 | 0.712 | 0.617 | |
| VSENetwork=ResNet2021.04 | 52 | 83.1 | 92 | — | — |