Image Captioning on COCO Caption
143.3CIDErSimVLM
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| SimVLMModel Size=Huge2021.08 | 143.3 | 40.6 | 33.7 | 25.4 | |
| SimVLMModel Size=Large2021.08 | 142.6 | 40.3 | 33.4 | 24.7 | |
| VinVLModel Size=Large2021.08 | 140.9 | 41 | 31.1 | 25.2 | |
| UNITERModel Size=Large2021.08 | 140 | 41.7 | 30.6 | 24.5 | |
| LION-12Bparameters=12B2023.11 | 139.25 | — | — | — | |
| InstructBLIP (T5XXL)backbone=T5XXL2023.11 | 138.63 | — | — | — | |
| InstructBLIP+ (T5XXL)backbone=T5XXL, uses_in-house_data=true2023.11 | 138.28 | — | — | — | |
| InstructBLIP+ (T5XL)backbone=T5XL, uses_in-house_data=true2023.11 | 138.21 | — | — | — | |
| LION-4Bparameters=4B2023.11 | 138.2 | — | — | — | |
| mPLUG-2#PT Data=17M2023.02 | 137.7 | 41.6 | 30.9 | 23.7 | |
| BLIP#PT Data=129M2023.02 | 136.7 | 40.4 | — | — | |
| LEMONlarge#PT Data=200M2023.02 | 135.7 | 40.6 | 30.4 | 23.5 | |
| InstructBLIP (T5XL)backbone=T5XL2023.11 | 135.47 | — | — | — | |
| SimVLMModel Size=Base2021.08 | 134.8 | 39 | 32.9 | 24 | |
| VinVL#PT Data=5.65M2023.02 | 130.8 | 38.5 | 30.4 | — | |
| MiniGPTV2evaluated_by_authors=true2023.11 | 129.16 | — | — | — | |
| UNIMOModel Size=Large2021.08 | 127.7 | — | — | — | |
| MH-MoEExperts Number=82024.04 | 124.1 | 39.7 | 33.1 | 23 | |
| X-MoEExperts Number=82024.04 | 122.9 | 38.1 | 30.2 | 21.3 | |
| Adapter V22023.11 | 122.2 | — | — | — | |
| LLaVA-NeXT+CALLLM=Vicuna-13B, Resolution setting=High resolution setting2024.05 | 120.6 | — | — | — | |
| Dense2024.04 | 120.5 | 35.9 | 29.3 | 19.6 | |
| LLaVA-1.5+CALLLM=Vicuna-13B, Resolution setting=Low resolution setting2024.05 | 119.4 | — | — | — | |
| LLaVA-NeXTLLM=Vicuna-13B, Resolution setting=High resolution setting, Win/All=4/42024.05 | 118.5 | — | — | — | |
| Shikra2023.11 | 117.5 | — | — | — | |
| E2E-VLP#PT Data=4M2023.02 | 117.3 | 36.2 | — | — | |
| Unified VLP2021.12 | 116.9 | 36.5 | 28.4 | 21.2 | |
| Uni-PerceiverTuning protocol=fine-tuning, Percentage of data used in tuning=100%2021.12 | 116.5 | 35.6 | 28.1 | 21.5 | |
| VL-T5Model Size=Base2021.08 | 116.5 | — | — | — | |
| LLaVA-1.5LLM=Vicuna-13B, Resolution setting=Low resolution setting, Win/All=3/42024.05 | 115.9 | — | — | — | |
| LLaVA-NeXT+CALLLM=Vicuna-7B, Resolution setting=High resolution setting2024.05 | 114.7 | — | — | — | |
| Uni-PerceiverTuning protocol=prompt-tuning, Percentage of data used in tuning=10%2021.12 | 114.1 | 35 | 27.9 | 21.3 | |
| LLaVA-NeXTLLM=Vicuna-7B, Resolution setting=High resolution setting, Win/All=4/42024.05 | 112 | — | — | — | |
| LLaVA-1.5+CALLLM=Vicuna-7B, Resolution setting=Low resolution setting2024.05 | 111.8 | — | — | — | |
| LLaVA-1.5LLM=Vicuna-7B, Resolution setting=Low resolution setting, Win/All=4/42024.05 | 111.1 | — | — | — | |
| Encoder-Decoder#PT Data=12M2023.02 | 110.9 | — | — | — | |
| Uni-PerceiverTuning protocol=none2021.12 | 109.8 | 33.6 | 27 | 20.3 | |
| Uni-PerceiverTuning protocol=prompt-tuning, Percentage of data used in tuning=1%2021.12 | 109.6 | 34.3 | 27.2 | 21.2 | |
| Uni-PerceiverTuning protocol=fine-tuning, Percentage of data used in tuning=10%2021.12 | 109 | 32.7 | 27.5 | 21.1 | |
| Uni-PerceiverTuning protocol=fine-tuning, Percentage of data used in tuning=1%2021.12 | 100.1 | 28 | 26.8 | 20.2 | |
| Kosmos-12023.11 | 84.7 | — | — | — | |
| Flamingo-9Bparameters=9B2023.11 | 79.4 | — | — | — | |
| LLaVAevaluated_by_authors=true2023.11 | 73.85 | — | — | — | |
| Flamingo-3Bparameters=3B2023.11 | 73 | — | — | — | |
| MGM-HDLLM=Vicuna-7B, Resolution setting=High resolution setting, Win/All=3/42024.05 | 33.4 | — | — | — | |
| MGM-HD+CALLLM=Vicuna-7B, Resolution setting=High resolution setting2024.05 | 31.3 | — | — | — | |
| MGM-HD+CALLLM=Vicuna-13B, Resolution setting=High resolution setting2024.05 | 30 | — | — | — | |
| MGM+CALLLM=Gemma-2B, Resolution setting=Low resolution setting2024.05 | 29.7 | — | — | — | |
| MGM-HDLLM=Vicuna-13B, Resolution setting=High resolution setting, Win/All=3/42024.05 | 22.2 | — | — | — | |
| MGM+CALLLM=Vicuna-7B, Resolution setting=Low resolution setting2024.05 | 21.8 | — | — | — | |
| MGM+CALLLM=Vicuna-13B, Resolution setting=Low resolution setting2024.05 | 18.6 | — | — | — | |
| MGMLLM=Vicuna-7B, Resolution setting=Low resolution setting, Win/All=4/42024.05 | 18.2 | — | — | — | |
| MGMLLM=Vicuna-13B, Resolution setting=Low resolution setting, Win/All=4/42024.05 | 17.6 | — | — | — | |
| MiniGPT4evaluated_by_authors=true2023.11 | 17.04 | — | — | — | |
| MGMLLM=Gemma-2B, Resolution setting=Low resolution setting, Win/All=4/42024.05 | 8 | — | — | — |