Image Captioning on COCO (Karpathy split)
155.1CIDErmPLUG
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| mPLUG#Param.=0.6B, #Images=14M, Model Scale=Large-sized, Training Strategy=SCST2022.05 | 155.1 | — | 32 | 26 | 46.5 | — | |
| OFA#Param.=0.9B, #Images=54M, Model Scale=Huge/Giant-sized, Training Strategy=SCST2022.05 | 154.9 | — | 32.5 | 26.6 | 44.9 | — | |
| GIT2#Param.=5.1B, #Images=10.5B, Model Scale=Huge/Giant-sized, Training Strategy=SCST2022.05 | 152.7 | — | 32.2 | 26.4 | 44 | — | |
| GIT#Param.=0.7B, #Images=0.8B, Model Scale=Huge/Giant-sized, Training Strategy=SCST2022.05 | 151.1 | — | 32.2 | 26.3 | 44.1 | — | |
| UniversalCap_L#Param.=0.5B+0.3B, #Images=36M, Model Scale=Large-sized, Training Strategy=SCST2022.05 | 150.2 | — | 31.5 | 25.2 | 42.9 | — | |
| UniversalCap_B#Param.=0.2B+0.3B, #Images=36M, Model Scale=Base-sized, Training Strategy=SCST2022.05 | 149.7 | — | 31.4 | 25 | 42.9 | — | |
| PaLIPre-train (# Pairs)=1.6B, Evaluation Setting=Generative2023.03 | 149.1 | — | — | — | — | — | |
| LEMON_huge#Param.=0.7B+0.2B, #Images=0.2B, Model Scale=Huge/Giant-sized, Training Strategy=SCST2022.05 | 145.5 | — | 31.4 | 25.5 | 42.6 | — | |
| OFAOptimization=CE2022.05 | 145.3 | — | — | — | — | — | |
| OFA#Param.=0.9B, #Images=54M, Model Scale=Huge/Giant-sized, Training Strategy=Cross-Entropy2022.05 | 145.3 | 43.9 | 31.8 | 24.8 | — | — | |
| GIT-2Pre-train (# Pairs)=12.9B, Evaluation Setting=Generative2023.03 | 145 | 44.1 | 31.4 | 24.8 | — | — | |
| GIT2#Param.=5.1B, #Images=10.5B, Model Scale=Huge/Giant-sized, Training Strategy=Cross-Entropy2022.05 | 145 | 44.1 | 31.4 | 24.8 | — | — | |
| GITPre-train (# Pairs)=0.8B, Evaluation Setting=Generative2023.03 | 144.8 | 44.1 | 31.5 | 24.7 | — | — | |
| GITOptimization=CE2022.05 | 144.8 | — | — | — | — | — | |
| GIT#Param.=0.7B, #Images=0.8B, Model Scale=Huge/Giant-sized, Training Strategy=Cross-Entropy2022.05 | 144.8 | 44.1 | 31.5 | 24.7 | — | — | |
| GIT_L#Param.=0.3B, #Images=14M, Model Scale=Large-sized, Training Strategy=SCST2022.05 | 144.6 | — | 31.2 | 25.4 | 42.3 | — | |
| CoCaPre-train (# Pairs)=4.8B2023.03 | 143.6 | 40.9 | 33.9 | 24.7 | — | — | |
| CoCaOptimization=CE2022.05 | 143.6 | — | — | — | — | — | |
| CoCa#Param.=2.1B, #Images=4.8B, Model Scale=Huge/Giant-sized, Training Strategy=Cross-Entropy2022.05 | 143.6 | 40.9 | 33.9 | 24.7 | — | — | |
| SimVLMModel Size=Huge, Pre-train (# Pairs)=1.8B2023.03 | 143.3 | 40.6 | 33.7 | 25.4 | — | — | |
| SimVLMOptimization=CE2022.05 | 143.3 | — | — | — | — | — | |
| SimVLM_Huge#Images=1.8B, Model Scale=Huge/Giant-sized, Training Strategy=Cross-Entropy2022.05 | 143.3 | 40.6 | 33.7 | 25.4 | — | — | |
| mPLUG#Param.=0.6B, #Images=14M, Model Scale=Large-sized, Training Strategy=Cross-Entropy2022.05 | 141 | 43.1 | 31.4 | 24.2 | — | — | |
| VinVL_L#Param.=0.3B+0.2B, #Images=6M, Model Scale=Large-sized, Training Strategy=SCST2022.05 | 140.9 | — | 31.1 | 25.2 | 41 | — | |
| VinVL_B#Param.=0.1B+0.2B, #Images=6M, Model Scale=Base-sized, Training Strategy=SCST2022.05 | 140.4 | — | 30.9 | 25.1 | 40.9 | — | |
| OSCAR_L#Param.=0.3B+64M, #Images=4M, Model Scale=Large-sized, Training Strategy=SCST2022.05 | 140 | — | 30.6 | 24.5 | 41.7 | — | |
| LEMONModel Size=Huge, Pre-train (# Pairs)=200M2023.03 | 139.1 | 41.5 | 30.8 | 24.1 | — | — | |
| LEMONOptimization=CE2022.05 | 139.1 | — | — | — | — | — | |
| GIT_B#Param.=0.1B, #Images=4M, Model Scale=Base-sized, Training Strategy=SCST2022.05 | 139.1 | — | 30.4 | 24.3 | 41.3 | — | |
| LEMON_huge#Param.=0.7B+0.2B, #Images=0.2B, Model Scale=Huge/Giant-sized, Training Strategy=Cross-Entropy2022.05 | 139.1 | 41.5 | 30.8 | 24.1 | — | — | |
| GITModel Size=Large, Pre-train (# Pairs)=20M, Evaluation Setting=Generative2023.03 | 138.5 | 42 | 30.8 | 23.8 | — | — | |
| GIT_L#Param.=0.3B, #Images=14M, Model Scale=Large-sized, Training Strategy=Cross-Entropy2022.05 | 138.5 | 42 | 30.8 | 23.8 | — | — | |
| FlamingoOptimization=CE2022.05 | 138.1 | — | — | — | — | — | |
| ViTCap#Param.=0.2B, #Images=4M, Model Scale=Base-sized, Training Strategy=SCST2022.05 | 138.1 | — | 30.1 | 24.1 | 41.2 | — | |
| Flamingo#Param.=80B, #Images=2.3B, Model Scale=Huge/Giant-sized, Training Strategy=Cross-Entropy2022.05 | 138.1 | — | — | — | — | — | |
| OSCAR_B#Param.=0.1B+64M, #Images=4M, Model Scale=Base-sized, Training Strategy=SCST2022.05 | 137.6 | — | 29.7 | 22.8 | 40.5 | — | |
| BLIPModel Size=Large, Pre-train (# Pairs)=129M, Evaluation Setting=Generative2023.03 | 136.7 | 40.4 | — | — | — | — | |
| BLIP_ViT-L#Images=129M, Model Scale=Large-sized, Training Strategy=Cross-Entropy2022.05 | 136.7 | 40.4 | — | — | — | — | |
| PrismerModel Size=Large, Pre-train (# Pairs)=12.7M, Evaluation Setting=Generative2023.03 | 136.5 | 40.4 | 31.4 | 24.4 | — | — | |
| LEMONModel Size=Large, Pre-train (# Pairs)=200M2023.03 | 135.7 | 40.6 | 30.4 | 23.5 | — | — | |
| PrismerZModel Size=Large, Pre-train (# Pairs)=12.7M, Evaluation Setting=Generative2023.03 | 135.7 | 40 | 31.2 | 24.2 | — | — | |
| PrismerModel Size=Base, Pre-train (# Pairs)=12.7M, Evaluation Setting=Generative2023.03 | 135.1 | 40.1 | 31.1 | 24.1 | — | — | |
| PrismerZModel Size=Base, Pre-train (# Pairs)=12.7M, Evaluation Setting=Generative2023.03 | 133.7 | 39.7 | 31.1 | 24.1 | — | — | |
| BLIPModel Size=Base, Pre-train (# Pairs)=129M, Evaluation Setting=Generative2023.03 | 133.3 | 39.7 | — | — | — | — | |
| LEMONModel Size=Base, Pre-train (# Pairs)=200M2023.03 | 133.3 | 40.3 | 30.2 | 23.3 | — | — | |
| MiniVLM#Param.=46M+8M, #Images=11M, Model Scale=Tiny-sized, Training Strategy=SCST2022.05 | 131.7 | — | 29.7 | 23.5 | 39.2 | — | |
| GITModel Size=Base, Pre-train (# Pairs)=10M, Evaluation Setting=Generative2023.03 | 131.4 | 40.4 | 30 | 23 | — | — | |
| GIT_B#Param.=0.1B, #Images=4M, Model Scale=Base-sized, Training Strategy=Cross-Entropy2022.05 | 131.4 | 40.4 | 30 | 23 | — | — | |
| UFOOptimization=CE2022.05 | 131.2 | — | — | — | — | — | |
| UFO_L#Param.=0.3B, #Images=4M, Model Scale=Large-sized, Training Strategy=Cross-Entropy2022.05 | 131.2 | 38.7 | 30 | 23.3 | — | — | |
| VinVLModel Size=Large, Pre-train (# Pairs)=8.9M2023.03 | 130.8 | 38.5 | 30.4 | 23.4 | — | — | |
| VinVLOptimization=CE2022.05 | 130.8 | — | — | — | — | — | |
| VinVL_L#Param.=0.3B+0.2B, #Images=6M, Model Scale=Large-sized, Training Strategy=Cross-Entropy2022.05 | 130.8 | 38.5 | 30.4 | 23.4 | — | — | |
| VinVLModel Size=Base, Pre-train (# Pairs)=8.9M2023.03 | 129.3 | 38.2 | 30.3 | 23.6 | — | — | |
| VinVL_B#Param.=0.1B+0.2B, #Images=6M, Model Scale=Base-sized, Training Strategy=Cross-Entropy2022.05 | 129.3 | 38.2 | 30.3 | 23.6 | — | — | |
| OSCARModel Size=Large, Pre-train (# Pairs)=6.5M2023.03 | 127.8 | 37.4 | 30.7 | 23.5 | — | — | |
| OSCAROptimization=CE2022.05 | 127.8 | — | — | — | — | — | |
| OSCAR_L#Param.=0.3B+64M, #Images=4M, Model Scale=Large-sized, Training Strategy=Cross-Entropy2022.05 | 127.8 | 37.4 | 30.7 | 23.5 | — | — | |
| ViTCapOptimization=CE2022.05 | 125.2 | — | — | — | — | — | |
| ViTCap#Param.=0.2B, #Images=4M, Model Scale=Base-sized, Training Strategy=Cross-Entropy2022.05 | 125.2 | 36.3 | 29.3 | 22.6 | — | — | |
| OSCARModel Size=Base, Pre-train (# Pairs)=6.5M2023.03 | 123.7 | 36.5 | 30.3 | 23.1 | — | — | |
| OSCAR_B#Param.=0.1B+64M, #Images=4M, Model Scale=Base-sized, Training Strategy=Cross-Entropy2022.05 | 123.7 | 36.5 | 30.3 | 23.1 | — | — | |
| UFO_B#Param.=0.1B, #Images=4M, Model Scale=Base-sized, Training Strategy=Cross-Entropy2022.05 | 122.8 | 36 | 28.9 | 22.2 | — | — | |
| DistillVLMOptimization=CE2022.05 | 120.8 | — | — | — | — | — | |
| DistillVLM#Param.=46M+8M, #Images=4M, Model Scale=Tiny-sized, Training Strategy=Cross-Entropy2022.05 | 120.8 | 35.6 | 28.7 | 22.1 | — | — | |
| MiniVLMOptimization=CE2022.05 | 119.8 | — | — | — | — | — | |
| MiniVLM#Param.=46M+8M, #Images=11M, Model Scale=Tiny-sized, Training Strategy=Cross-Entropy2022.05 | 119.8 | 35.6 | 28.6 | 21.6 | — | — | |
| SCD-Net2024.12 | 118 | 37.3 | 28.1 | 21.6 | — | 58 | |
| CrossFlow2024.12 | 116.2 | 36.4 | 27.8 | 20.4 | — | 57.1 | |
| NAIC-CMAL2024.12 | 115.5 | 35.3 | 27.3 | 20.8 | — | 56.9 | |
| CrossFlow-T2Iinversion=true2024.12 | 111.2 | 33.1 | 27 | 20.3 | — | 56.4 | |
| SATIC2024.12 | 111 | 32.9 | 27 | 20.5 | — | — | |
| MIR2024.12 | 109.5 | 32.5 | 27.2 | 20.6 | — | — | |
| MNIC2024.12 | 108.1 | 30.9 | 27.5 | 21 | — | 55.6 |