Image Captioning on TextCaps
164.3CIDErPaLI-3
Evaluation Results
| Method | Links | |
|---|---|---|
| PaLI-3OCR pipeline input=true, Resolution=812x8122023.10 | 164.3 | |
| PaLI-XOCR pipeline input=true, Resolution=812x8122023.10 | 163.7 | |
| PaLIInput type=Pipelined2022.10 | 160.4 | |
| PaLI-3OCR pipeline input=false, Resolution=812x8122023.10 | 158.8 | |
| PaLI-XOCR pipeline input=false, Resolution=812x8122023.10 | 147 | |
| GIT2Input type=Pixel-only, Additional labeled data=true2022.10 | 145 | |
| GenLIPArch=g/16, Data=8.0B2026.05 | 135.4 | |
| SigLIP2Arch=g/16, Data=40.0B2026.05 | 134.5 | |
| DocOwl2Size=8B, Visual tokens=3242024.09 | 131.8 | |
| DocOwl 1.5Size=8B, Visual tokens=16982024.09 | 131.6 | |
| SigLIP2Arch=So/16, Data=40.0B2026.05 | 131.5 | |
| GenLIPArch=L/16, Data=8.0B2026.05 | 131.4 | |
| GenLIPArch=So/16, Data=8.0B2026.05 | 129.5 | |
| LLaVA-1.5 + QA-ViTObserved in training=true2024.02 | 128.7 | |
| SigLIP2Arch=L/16, Data=40.0B2026.05 | 127.8 | |
| OVision2Arch=L/16, Data=12.8B2026.05 | 127.4 | |
| LLaVA-1.5Observed in training=true2024.02 | 126.4 | |
| InstructBLIPObserved in training=true, OCR tokens=true2024.02 | 126 | |
| LLaVA-NeXT+CALLLM=Vicuna-7B, Resolution setting=High resolution setting2024.05 | 124.7 | |
| LLaVA-NeXT+CALLLM=Vicuna-13B, Resolution setting=High resolution setting2024.05 | 124.4 | |
| AIMv2Arch=L/14, Data=12.0B2026.05 | 122.4 | |
| SigLIPArch=L/16, Data=40.0B2026.05 | 120.7 | |
| UReaderSize=7B, Visual tokens=8412024.09 | 118.4 | |
| LLaVA-NeXTLLM=Vicuna-13B, Resolution setting=High resolution setting, Win/All=4/42024.05 | 118.2 | |
| CLIPArch=L/14, Data=12.8B2026.05 | 117.9 | |
| LLaVA-NeXTLLM=Vicuna-7B, Resolution setting=High resolution setting, Win/All=4/42024.05 | 115 | |
| mPLUG-DocOwlObserved in training=true2024.02 | 111.9 | |
| DocOwlSize=7B, Visual tokens=8412024.09 | 111.9 | |
| LION-12Bparameters=12B2023.11 | 108.76 | |
| LLaVA-1.5+CALLLM=Vicuna-7B, Resolution setting=Low resolution setting2024.05 | 106.7 | |
| LBRBackbone=LLaVA-NEXT-3B2026.05 | 105.8 | |
| LLaVA-1.5+CALLLM=Vicuna-13B, Resolution setting=Low resolution setting2024.05 | 105.6 | |
| InstructBLIP (T5XXL)backbone=T5XXL2023.11 | 105.44 | |
| LBRBackbone=LLaVA-1.5-13B2026.05 | 105.2 | |
| LION-4Bparameters=4B2023.11 | 104.87 | |
| InstructBLIP (T5XL)backbone=T5XL2023.11 | 104.17 | |
| LLaVA-NEXT-3BBackbone=LLaVA-NEXT-3B2026.05 | 104 | |
| LLaVA-1.5-13BBackbone=LLaVA-1.5-13B2026.05 | 103.9 | |
| LLaVA-1.5LLM=Vicuna-13B, Resolution setting=Low resolution setting, Win/All=3/42024.05 | 102.4 | |
| LLaVA-1.5LLM=Vicuna-7B, Resolution setting=Low resolution setting, Win/All=4/42024.05 | 100.7 | |
| VanillaModel=LLaVA-1.5-7B, Retention Ratio=50%2026.06 | 100.3 | |
| VanillaModel=LLaVA-1.5-7B, Retention Ratio=20%2026.06 | 100.3 | |
| VanillaModel=LLaVA-1.5-7B, Retention Ratio=10%2026.06 | 100.3 | |
| TGV-KVModel=LLaVA-1.5-7B, Retention Ratio=50%2026.06 | 99.8 | |
| TAPInput type=Pipelined2022.10 | 99.5 | |
| LBRBackbone=LLaVA-1.5-7B2026.05 | 99.1 | |
| LLaVA-1.5-7BBackbone=LLaVA-1.5-7B2026.05 | 98.4 | |
| MM1-30BShot=16, Model Scale=30B2024.03 | 97.6 | |
| MM1-30BShot=162024.03 | 97.6 | |
| SnapKVModel=LLaVA-1.5-7B, Retention Ratio=50%2026.06 | 97.1 | |
| Pix2Struct-LargeInput type=Pixel-only, Scale=Large2022.10 | 95.5 | |
| Pix2Struct-largeSize=1B, Visual tokens=2048, Fine-tuned=true2024.09 | 95.5 | |
| PrefixKVModel=LLaVA-1.5-7B, Retention Ratio=50%2026.06 | 94.3 | |
| MonkeyParameters=9.8B, Resolution=896x8962023.11 | 93.2 | |
| MonkeySize=9B, Visual tokens=12802024.09 | 93.2 | |
| MM1-7BShot=162024.03 | 93.1 | |
| MM1-30BShot=8, Model Scale=30B2024.03 | 92.9 | |
| MM1-30BShot=82024.03 | 92.9 | |
| MM1-3BShot=162024.03 | 91.6 | |
| MM1-3BShot=8, Model Scale=3B2024.03 | 88.8 | |
| MM1-3BShot=82024.03 | 88.8 | |
| MM1-7BShot=8, Model Scale=7B2024.03 | 88.2 | |
| MM1-7BShot=82024.03 | 88.2 | |
| Pix2Struct-BaseInput type=Pixel-only, Scale=Base2022.10 | 88 | |
| Pix2Struct-baseSize=<1B, Visual tokens=2048, Fine-tuned=true2024.09 | 88 | |
| TGV-KVModel=LLaVA-1.5-7B, Retention Ratio=20%2026.06 | 87.8 | |
| MM1-30BShot=42024.03 | 87.5 | |
| MM1-7BShot=42024.03 | 84.5 | |
| MM1-3BShot=42024.03 | 84.1 | |
| AIMv2architecture=ViT-3B/14, #patches=576, resolution=336px2024.11 | 83.8 | |
| AIMv2architecture=ViT-1B/14, #patches=576, resolution=336px2024.11 | 82.6 | |
| InstructBLIP+ (T5XL)backbone=T5XL, uses_in-house_data=true2023.11 | 82.55 | |
| InstructBLIP+ (T5XXL)backbone=T5XXL, uses_in-house_data=true2023.11 | 82.53 | |
| SnapKVModel=LLaVA-1.5-7B, Retention Ratio=20%2026.06 | 82.3 | |
| IDEFICS-80BShot=16, Model Scale=80B2024.03 | 81.4 | |
| IDEFICS-80BShot=162024.03 | 81.4 | |
| AIMv2architecture=ViT-L/14, #patches=576, resolution=336px2024.11 | 81.1 | |
| M4CInput type=Pipelined2022.10 | 81 | |
| AIMv2architecture=ViT-H/14, #patches=576, resolution=336px2024.11 | 80.7 | |
| MiniGPTV2evaluated_by_authors=true2023.11 | 80.6 | |
| StreamingLLMModel=LLaVA-1.5-7B, Retention Ratio=50%2026.06 | 78.4 | |
| IDEFICS-80BShot=8, Model Scale=80B2024.03 | 77.6 | |
| IDEFICS-80BShot=82024.03 | 77.6 | |
| InstructBLIPObserved in training=true2024.02 | 75.6 | |
| OpenAI CLIParchitecture=ViT-L/14, #patches=5762024.11 | 75.3 | |
| DonutInput type=Pixel-only2022.10 | 74.4 | |
| IDEFICS-80BShot=42024.03 | 72.7 | |
| SigLIParchitecture=ViT-So400M/14, #patches=7522024.11 | 70.8 | |
| BLIP22024.02 | 70.4 | |
| SigLIParchitecture=ViT-L/14, #patches=5762024.11 | 69.9 | |
| IDEFICS-9BShot=162024.03 | 67.4 | |
| Qwen-VLBackbone=Qwen-7B2023.11 | 65.1 | |
| MM1-30BShot=0, Model Scale=30B2024.03 | 64.9 | |
| MM1-30BShot=02024.03 | 64.9 | |
| MM1-7BShot=0, Model Scale=7B2024.03 | 64.2 | |
| MM1-7BShot=02024.03 | 64.2 | |
| TGV-KVModel=LLaVA-1.5-7B, Retention Ratio=10%2026.06 | 63.6 | |
| MM1-3BShot=0, Model Scale=3B2024.03 | 63.3 | |
| MM1-3BShot=02024.03 | 63.3 | |
| IDEFICS-9BShot=8, Model Scale=9B2024.03 | 63.2 |