Image Captioning on NoCaps 1.0 (val)
127Overall ScorePaLI
Evaluation results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| PaLIPre-train (# Pairs)=1.6B, Evaluation Setting=Generative2023.03 | 127 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GIT-2Pre-train (# Pairs)=12.9B, Evaluation Setting=Generative2023.03 | 126.9 | 126.9 | 125.8 | 130.6 | — | — | — | — | — | — | — | — | — | — | |
| GITPre-train (# Pairs)=0.8B, Evaluation Setting=Generative2023.03 | 125.5 | 129.8 | 124.1 | 127.1 | — | — | — | — | — | — | — | — | — | — | |
| CoCaPre-train (# Pairs)=4.8B2023.03 | 122.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LEMONModel Size=Huge, Pre-train (# Pairs)=200M2023.03 | 117.3 | 118 | 116.3 | 120.2 | — | — | — | — | — | — | — | — | — | — | |
| LEMONModel Size=Large, Pre-train (# Pairs)=200M2023.03 | 113.4 | 116.9 | 113.3 | 111.3 | — | — | — | — | — | — | — | — | — | — | |
| BLIPModel Size=Large, Pre-train (# Pairs)=129M, Evaluation Setting=Generative2023.03 | 113.2 | 114.9 | 112.1 | 115.3 | — | — | — | — | — | — | — | — | — | — | |
| PrismerModel Size=Large, Pre-train (# Pairs)=12.7M, Evaluation Setting=Generative2023.03 | 112.9 | 114.2 | 112.5 | 113.5 | — | — | — | — | — | — | — | — | — | — | |
| SimVLMhugeSetup=pretrain-finetune2021.08 | 112.2 | 113.7 | 110.9 | 115.2 | — | — | — | — | — | — | — | — | — | — | |
| SimVLMModel Size=Huge, Pre-train (# Pairs)=1.8B2023.03 | 112.2 | 113.7 | 110.9 | 115.2 | — | — | — | — | — | — | — | — | — | — | |
| PrismerZModel Size=Large, Pre-train (# Pairs)=12.7M, Evaluation Setting=Generative2023.03 | 111.8 | 112.3 | 111.2 | 112.8 | — | — | — | — | — | — | — | — | — | — | |
| SimVLMhugeSetup=few-shot2021.08 | 110.4 | 111.8 | 110.6 | 111 | — | — | — | — | — | — | — | — | — | — | |
| BLIPModel Size=Base, Pre-train (# Pairs)=129M, Evaluation Setting=Generative2023.03 | 109.6 | 111.8 | 108.6 | 111.5 | — | — | — | — | — | — | — | — | — | — | |
| PrismerModel Size=Base, Pre-train (# Pairs)=12.7M, Evaluation Setting=Generative2023.03 | 109.1 | 108.8 | 108.3 | 111.7 | — | — | — | — | — | — | — | — | — | — | |
| PrismerZModel Size=Base, Pre-train (# Pairs)=12.7M, Evaluation Setting=Generative2023.03 | 107.5 | 108.7 | 107.8 | 105.8 | — | — | — | — | — | — | — | — | — | — | |
| GITModel Size=Large, Pre-train (# Pairs)=20M, Evaluation Setting=Generative2023.03 | 106.9 | 107.7 | 107.8 | 102.5 | — | — | — | — | — | — | — | — | — | — | |
| LEMONModel Size=Base, Pre-train (# Pairs)=200M2023.03 | 106.8 | 107.7 | 106.2 | 107.9 | — | — | — | — | — | — | — | — | — | — | |
| SimVLMlargeSetup=few-shot2021.08 | 102.2 | 102.5 | 100.9 | 106 | — | — | — | — | — | — | — | — | — | — | |
| SimVLMhugeSetup=zero-shot2021.08 | 101.4 | 101.2 | 100.4 | 102.3 | — | — | — | — | — | — | — | — | — | — | |
| SimVLMlargeSetup=zero-shot2021.08 | 96.6 | 97.6 | 96.5 | 96.3 | — | — | — | — | — | — | — | — | — | — | |
| GITModel Size=Base, Pre-train (# Pairs)=10M, Evaluation Setting=Generative2023.03 | 96.6 | 100.7 | 97.7 | 89.6 | — | — | — | — | — | — | — | — | — | — | |
| VinVLSetup=pretrain-finetune, CIDEr optimization=true2021.08 | 94.3 | 103.7 | 95.6 | 83.8 | — | — | — | — | — | — | — | — | — | — | |
| VinVLModel Size=Base, Pre-train (# Pairs)=8.9M2023.03 | 94.3 | 103.7 | 95.6 | 83.8 | — | — | — | — | — | — | — | — | — | — | |
| SimVLMbaseSetup=few-shot2021.08 | 93.7 | 95 | 91.9 | 98.5 | — | — | — | — | — | — | — | — | — | — | |
| SimVLMbaseSetup=zero-shot2021.08 | 83.5 | 83.2 | 84.1 | 82.5 | — | — | — | — | — | — | — | — | — | — | |
| OSCARSetup=pretrain-finetune2021.08 | 83.4 | 85.4 | 84 | 80.3 | — | — | — | — | — | — | — | — | — | — | |
| OSCARModel Size=Large, Pre-train (# Pairs)=6.5M2023.03 | 83.4 | 85.4 | 84 | 80.3 | — | — | — | — | — | — | — | — | — | — | |
| OSCARModel Size=Base, Pre-train (# Pairs)=6.5M2023.03 | 81.1 | 83.4 | 81.6 | 77.6 | — | — | — | — | — | — | — | — | — | — | |
| M2 TransformerSetup=supervised, CIDEr optimization=true2021.08 | 75 | 81.2 | — | 69.4 | — | — | — | — | — | — | — | — | — | — | |
| BLS-InfoBatchPruned %=20.92026.04 | 65.3 | 58.8 | 63.1 | 65.8 | — | — | — | — | — | — | — | — | — | — | |
| BLS-SeTaPruned %=28.12026.04 | 65.3 | 58.9 | 63.1 | 65.6 | — | — | — | — | — | — | — | — | — | — | |
| Full2026.04 | 65.2 | 58.4 | 63.1 | 65.3 | — | — | — | — | — | — | — | — | — | — | |
| BLIP#Trainable Params=583M, Open-sourced?=true, Zero-shot=true2023.01 | — | — | — | — | 113.2 | 14.8 | — | — | — | — | — | — | — | — | |
| BLIP-2#Trainable Params=188M, Open-sourced?=true, Zero-shot=true2023.01 | — | — | — | — | 121.6 | 15.8 | — | — | — | — | — | — | — | — | |
| BLIPCapFilt-L#Pretrain Images=129M, CIDEr optimization=false2022.06 | — | — | — | — | 109.6 | 14.7 | — | — | — | — | — | — | — | — | |
| CoCaNumber of parameters=2.1B2023.05 | — | — | — | — | 122.4 | — | — | — | — | — | — | — | — | — | |
| FewVLMlearning_mode=zero-shot, model_scale=base2022.06 | — | — | — | — | 42.2 | 8.5 | — | — | — | — | — | — | — | — | |
| FIBER-B#Pretrain Images=4M, CIDEr optimization=false2022.06 | — | — | — | — | 88.6 | 13 | — | — | — | — | — | — | — | — | |
| FIBER-B#Pretrain Images=4M, CIDEr optimization=true2022.06 | — | — | — | — | 96.7 | 13.4 | — | — | — | — | — | — | — | — | |
| FIBER-GOLD-B#Pretrain Images=4M, CIDEr optimization=false2022.06 | — | — | — | — | 92.8 | 13.4 | — | — | — | — | — | — | — | — | |
| FIBER-GOLD-B#Pretrain Images=4M, CIDEr optimization=true2022.06 | — | — | — | — | 99.2 | 13.7 | — | — | — | — | — | — | — | — | |
| GITNumber of parameters=0.7B2023.05 | — | — | — | — | 125.5 | — | — | — | — | — | — | — | — | — | |
| GIT2Number of parameters=5.1B2023.05 | — | — | — | — | 126.9 | — | — | — | — | — | — | — | — | — | |
| LEMON-B#Pretrain Images=200M, CIDEr optimization=false2022.06 | — | — | — | — | 106.8 | 14.1 | — | — | — | — | — | — | — | — | |
| METALMlearning_mode=zero-shot, model_scale=base2022.06 | — | — | — | — | 58.7 | 8.6 | — | — | — | — | — | — | — | — | |
| PaLINumber of parameters=17B2023.05 | — | — | — | — | 127 | — | — | — | — | — | — | — | — | — | |
| PaLI-XNumber of parameters=55B2023.05 | — | — | — | — | 126.3 | — | — | — | — | — | — | — | — | — | |
| SimVLM2023.05 | — | — | — | — | 112.2 | — | — | — | — | — | — | — | — | — | |
| SimVLM#Trainable Params=1.4B, Open-sourced?=false, Zero-shot=true2023.01 | — | — | — | — | 112.2 | — | — | — | — | — | — | — | — | — | |
| UFO-B#Pretrain Images=4M, CIDEr optimization=false2022.06 | — | — | — | — | 80.7 | 12.5 | — | — | — | — | — | — | — | — | |
| ViECapPruned %=0%2025.03 | — | — | — | — | — | — | 58.4 | — | 63.1 | — | 65.3 | — | 65.2 | — | |
| ViECap + SeTaPruned %=30.7%2025.03 | — | — | — | — | — | — | 58.7 | — | 63.2 | — | 65.6 | — | 65.4 | — | |
| ViECap + SeTaPruned %=41.2%2025.03 | — | — | — | — | — | — | 57.7 | — | 61.9 | — | 64.6 | — | 64.1 | — | |
| ViECap + SeTaPruned %=51.9%2025.03 | — | — | — | — | — | — | 57 | — | 61.6 | — | 64.4 | — | 63.9 | — | |
| VinVL-B#Pretrain Images=5.7M, CIDEr optimization=true, Constrained beam search=true2022.06 | — | — | — | — | 94.3 | 13.1 | — | — | — | — | — | — | — | — | |
| VITCAP#Pretrain Images=4M, CIDEr optimization=true2022.06 | — | — | — | — | 89.2 | 12.7 | — | — | — | — | — | — | — | — | |
| VL-T5learning_mode=zero-shot, model_scale=base2022.06 | — | — | — | — | 4.4 | 5.3 | — | — | — | — | — | — | — | — |