Image-to-Sentence Retrieval on MS-COCO 5K full (test)
77Recall@1ALIGN
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| ALIGNPretrain=1.8B, GPUs=1024 TPUv3, Bs=16384, Zero-shot=false2024.04 | 77 | 93.5 | — | — | |
| ALBEFPretrain=4.0M, GPUs=8 A100, Bs=5122024.04 | 73.1 | 91.4 | — | — | |
| OSCARPretrain=6.5M, GPUs=16 V100, Bs=1,0242024.04 | 70 | 91.1 | — | — | |
| 3SHNetType=Region+Grid, Ensemble=true2024.04 | 67.9 | 90.5 | 95.4 | 471.2 | |
| 3SHNetGPUs=1 TitanX, Bs=2562024.04 | 67.9 | 90.5 | — | — | |
| 3SHNetType=Region+Grid, Ensemble=false2024.04 | 67.1 | 89.8 | 95.2 | 468 | |
| 3SHNetType=Grid, Ensemble=true2024.04 | 66.2 | 89.8 | 94.7 | 464.8 | |
| 3SHNetType=Region, Ensemble=true2024.04 | 65.3 | 88.8 | 94.1 | 460.2 | |
| 3SHNetType=Grid, Ensemble=false2024.04 | 64.1 | 88.9 | 94.3 | 459 | |
| VSE∞ w/ Seg.Type=Region+Grid, Ensemble=false2024.04 | 63.9 | 88.3 | 94.2 | 457.1 | |
| 3SHNetType=Region, Ensemble=false2024.04 | 63.8 | 88.1 | 94 | 454.9 | |
| Imp.Type=Region+Grid, Ensemble=true2024.04 | 63.5 | 87.9 | 93.5 | 452.9 | |
| UNITERPretrain=9.6M, GPUs=16 V100, Bs=1922024.04 | 63.3 | 87 | — | — | |
| VSE∞Type=Region+Grid, Ensemble=false2024.04 | 62.5 | 87.8 | 94 | 451.8 | |
| UnicoderPretrain=3.8M, GPUs=4 V1002024.04 | 62.3 | 87.1 | — | — | |
| CMSEIType=Region, Ensemble=true2024.04 | 61.5 | 86.3 | 92.7 | 441.2 | |
| CMCANType=Region+Grid, Ensemble=false2024.04 | 61.5 | — | 92.9 | — | |
| MKTLONType=Region, Ensemble=true2024.04 | 61.4 | 86.7 | 92.8 | 442.8 | |
| DCINType=Region, Ensemble=true2024.04 | 60.8 | 86.3 | 93 | 443 | |
| KIDRRType=Region, Ensemble=true2024.04 | 60.3 | 86.1 | 92.5 | 438 | |
| HGANType=Region+Grid, Ensemble=false2024.04 | 60 | 85.8 | 92.8 | 444.4 | |
| AMEType=Region, Ensemble=true2024.04 | 59.9 | 85.2 | 92.3 | 436.3 | |
| CHANType=Region, Ensemble=false2024.04 | 59.8 | 87.2 | 93.3 | 443.9 | |
| DIMEType=Region, Ensemble=true2024.04 | 59.3 | 85.4 | 91.9 | 435.8 | |
| VSE∞Type=Grid, Ensemble=false2024.04 | 59.1 | 85.9 | 92.8 | 440 | |
| NAAFType=Region, Ensemble=true2024.04 | 58.9 | 85.2 | 92 | 430.9 | |
| ALIGNPretrain=1.8B, GPUs=1024 TPUv3, Bs=16384, Zero-shot=true2024.04 | 58.6 | 83 | — | — | |
| CLIPPretrain=400M, GPUs=592 V100, Bs=32,768, Zero-shot=true2024.04 | 58.4 | 81.5 | — | — | |
| VSE∞Type=Region, Ensemble=false2024.04 | 58.3 | 85.3 | 92.3 | 434.3 | |
| VILBERTPretrain=3.3M, GPUs=8 TitanX, Bs=5122024.04 | 57.5 | 84 | — | — | |
| RCTRNType=Region, Ensemble=true2024.04 | 57.1 | 83.4 | 91.9 | 431.6 | |
| VSRN++Type=Region, Ensemble=true2024.04 | 54.7 | 82.9 | 90.9 | 425.4 | |
| IMRAMType=Region, Ensemble=true2024.04 | 53.7 | 83.2 | 91 | 416.5 | |
| CRGNType=Region+Grid, Ensemble=false2024.04 | 51.2 | 80.6 | 89.7 | 406.4 | |
| SCOType=Grid, Ensemble=false2024.04 | 42.8 | 72.3 | 83 | 369.6 |